-- Logs begin at Wed 2022-09-07 12:20:12 UTC, end at Wed 2022-09-07 13:54:27 UTC. -- Sep 07 13:06:56.203657 dv-cinder-856259-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 07 13:06:56.213747 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 07 13:06:56.215991 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 7 13:06:56 2022] *** Sep 07 13:06:56.215991 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 07 13:06:56.215991 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 07 13:06:56.215991 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: nodename: dv-cinder-856259-1-iscsi Sep 07 13:06:56.215991 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: machine: x86_64 Sep 07 13:06:56.215991 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: clock source: unix Sep 07 13:06:56.215991 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: pcre jit disabled Sep 07 13:06:56.215991 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: detected number of CPU cores: 4 Sep 07 13:06:56.215991 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: current working directory: / Sep 07 13:06:56.215991 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: detected binary path: /usr/bin/uwsgi-core Sep 07 13:06:56.215991 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: your processes number limit is 47680 Sep 07 13:06:56.215991 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: your memory page size is 4096 bytes Sep 07 13:06:56.215991 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: detected max file descriptor number: 2048 Sep 07 13:06:56.215991 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: lock engine: pthread robust mutexes Sep 07 13:06:56.216747 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: thunder lock: enabled Sep 07 13:06:56.216747 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 07 13:06:56.216747 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 07 13:06:56.257061 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: Python main interpreter initialized at 0x55affe43d9c0 Sep 07 13:06:56.257061 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: python threads support enabled Sep 07 13:06:56.257061 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: your server socket listen backlog is limited to 100 connections Sep 07 13:06:56.257061 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: your mercy for graceful operations on workers is 90 seconds Sep 07 13:06:56.257718 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: mapped 403077 bytes (393 KB) for 2 cores Sep 07 13:06:56.257718 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: *** Operational MODE: preforking *** Sep 07 13:06:56.257933 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: *** uWSGI is running in multiple interpreter mode *** Sep 07 13:06:56.257933 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: spawned uWSGI master process (pid: 105303) Sep 07 13:06:56.258049 dv-cinder-856259-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 07 13:06:56.259829 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: spawned uWSGI worker 1 (pid: 105315, cores: 1) Sep 07 13:06:56.259829 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: spawned uWSGI worker 2 (pid: 105316, cores: 1) Sep 07 13:06:56.259829 dv-cinder-856259-1-iscsi devstack@keystone.service[105303]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 07 13:07:00.815275 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 13:07:00.815275 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: from cryptography.utils import int_from_bytes Sep 07 13:07:00.816167 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 13:07:00.816167 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: from cryptography.utils import int_from_bytes Sep 07 13:07:00.840468 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 13:07:00.840468 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: from cryptography.utils import int_from_bytes Sep 07 13:07:00.841208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 13:07:00.841208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: from cryptography.utils import int_from_bytes Sep 07 13:07:00.869471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 07 13:07:00.869721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 07 13:07:00.869907 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] command line args: [] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 07 13:07:00.870115 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 07 13:07:00.870302 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ================================================================================ {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 07 13:07:00.870589 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] admin_token = **** {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.870842 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] config_dir = [] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.871009 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.871187 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] config_source = [] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.871491 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] debug = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.871617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.872299 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.872436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] insecure_debug = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.872696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.872949 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.873192 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] list_limit = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.873377 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] log_config_append = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.873559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.873742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] log_dir = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.873923 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] log_file = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.874166 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.874423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.874740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.874923 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.877861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.877861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.877861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.877861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.877861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.877861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.877861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.877861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.878315 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.878315 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] notification_format = cadf {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.878315 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.878315 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.878315 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] publish_errors = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.878582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.878644 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.878841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.879065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.879344 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.879542 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] standard_threads = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.879779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] strict_password_check = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.880010 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.880225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] use_eventlog = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.880369 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] use_journal = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.880547 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] use_json = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.880723 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] use_stderr = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.880901 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] use_syslog = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.881199 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] watch_log_file = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.882139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.882400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.882669 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.882990 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.883331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.883579 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.883836 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.884076 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.884328 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.884586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.884837 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.885076 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.885327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.885567 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.885817 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.886067 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.886307 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.886564 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.886822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.887098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.887337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.887591 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.887858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.888112 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.888395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.888703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.888982 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.889177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.889411 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.889663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.889913 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.890145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.890378 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.890627 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.890876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.891136 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.891389 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.891624 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.891865 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.892106 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.892342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.892576 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.892813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.893049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.893294 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.893540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.893772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.894011 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.894371 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.894549 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.894789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.895060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.895298 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.895538 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.895779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.896019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.896261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.896498 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.896774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.897028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.897267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.897507 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.897742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.898004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.898244 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.898478 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.898718 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.898956 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.899249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.899492 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.899742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.899994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.900301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.900564 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.900772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 07 13:07:00.900862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.900946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 07 13:07:00.901098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.enabled = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.901172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] command line args: [] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 07 13:07:00.901242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.901312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 07 13:07:00.901426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ================================================================================ {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 07 13:07:00.901512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.901667 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.901744 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] admin_token = **** {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.901908 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.901991 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] config_dir = [] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.902100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.902185 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.902288 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] config_source = [] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.902386 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.902462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] debug = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.902624 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.902709 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.902895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.903017 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.905610 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.905738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] insecure_debug = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.905838 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.905945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.906044 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.906148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.906244 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.906345 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] list_limit = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.906345 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] log_config_append = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.906532 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.906643 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.906746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.906843 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] log_dir = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.906843 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] log_file = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.907074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.907190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.907260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.907324 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.907442 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.907523 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.907640 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.907640 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.907864 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.907951 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.908025 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.908025 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.908217 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.908217 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.908379 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.908498 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.908584 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.908702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.908786 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.908865 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.908943 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.909061 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.909142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.909234 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.909376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.909459 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.909532 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.909652 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.909739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] notification_format = cadf {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.909833 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.909913 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.909997 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.909997 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] auth.external = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.910115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.62/identity {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.910233 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] auth.mapped = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.910361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] publish_errors = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.910438 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.910505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.910623 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.910697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.910815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] auth.password = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.910899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.911048 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] auth.token = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.911175 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.911257 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.911330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.911459 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] catalog.caching = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.911550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] standard_threads = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.911622 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.911739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] strict_password_check = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.911826 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.911926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.911926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] use_eventlog = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.912114 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.912190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] use_journal = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.912298 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.912440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] use_json = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.912546 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.912638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] use_stderr = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.912638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] use_syslog = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.912808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] credential.caching = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.912886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] watch_log_file = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 13:07:00.913119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] credential.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.913206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.913317 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.913430 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.913513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.913513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.913513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.913830 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.913908 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.914057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.914122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.914252 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.914328 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.914394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.914570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.914684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.914760 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.914848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.915009 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.915094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.915165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.915254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.915329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.915401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.915480 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.915567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.915641 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.915711 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.915802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] federation.caching = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.915881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.915989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.916067 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.916197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] federation.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.916318 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.916444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.916647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.916729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.916906 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.916986 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.917149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.917241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.917530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.917624 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.917732 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.917811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.917992 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.918080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.918265 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.918362 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.918519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.918599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.918795 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] identity.caching = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.918887 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.919035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.919114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.919249 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.919326 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.919486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.919566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.919722 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.919802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.919971 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.920049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] identity.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.920203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.920298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.920443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.920524 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.920677 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.920755 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.920927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.921006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.921160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.921244 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.921395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.921474 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.921630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.921710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.921863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.921940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.922103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.922180 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.922540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.922631 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.922631 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.922784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.922857 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.922954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.923122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.923203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.923390 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.923538 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.923669 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.923808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.923959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.924040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.924196 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.924300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.924431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.924557 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.924662 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.924740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.924902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.924989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.925142 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.925233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.925391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.925469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.925625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.925704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.925863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.925941 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.926104 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.926186 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.926343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.926426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.926580 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.926662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.926849 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.926934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.927133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.927220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.927403 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.927492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.927641 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.password = **** {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.927720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.928428 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.928543 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.928667 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.928954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.929051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.929149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.929392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.929507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.929646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.929796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.930037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.930117 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.930277 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.930358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.930515 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.930585 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.930770 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.930839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.931053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.931140 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.931297 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.931372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.931536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.931615 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.931778 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.931859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.932013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.932092 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.932277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.932360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.932507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.enabled = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.932587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.932742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.932820 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.932984 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.933068 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.933231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.933313 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.933474 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.933570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.933712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.933789 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.933946 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.934026 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.934517 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.934617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.934680 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.934747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.934817 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.934880 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.934939 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.935018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.935216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.935299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.935456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.935533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.935702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.935770 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.935948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.936035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.936192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.936285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.936425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.936515 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.936667 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.936754 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.936904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.936982 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.937141 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.937219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.937375 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.937461 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.937610 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.937689 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.937844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.937922 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.938084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.938212 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.938311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.938385 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.938565 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] policy.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.938643 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.938805 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.938923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.939069 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.939206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.939333 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.939415 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.939538 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] receipt.caching = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.939722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.939803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.939995 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.940075 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.940242 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.940322 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.940475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.940554 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] auth.external = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.940710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] auth.mapped = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.940787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.940952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] resource.caching = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.941067 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.941204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.941311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.941453 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] resource.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.941547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] auth.password = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.941694 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.941772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] auth.token = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.941945 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.942025 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.942197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.942275 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] catalog.caching = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.942436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] revoke.caching = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.942520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.942686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.942766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.942925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.943113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.943200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.943273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] role.cache_time = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.943427 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.943528 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] role.caching = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.943708 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] credential.caching = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.944277 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] role.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.944501 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] role.list_limit = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.944754 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.945004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.945105 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] credential.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.945255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.945382 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.945503 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.945669 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.945827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.947507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.947603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.947666 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.951444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.951629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.951729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.951793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.951865 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.951937 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.952007 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.952078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.952157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.952231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.952318 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.952395 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.952466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.952546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.952546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.952687 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.952774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.952864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.952984 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.953073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.953148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] federation.caching = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.953220 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.953280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.953372 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.953480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] federation.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.953553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.953628 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.953689 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.953793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.953955 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.954065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.954205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.954340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.954467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.954586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.954721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.954851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.955025 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.955025 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.955264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.955396 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.955531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.955531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.955797 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.955941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] identity.caching = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.956073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.956205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.956338 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.956338 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.956699 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.956845 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] token.cache_time = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.956980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.957113 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] token.caching = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.957247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.957379 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.957572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] identity.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.957702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] token.provider = fernet {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.957831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.957962 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.958104 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.958219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.958363 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.958494 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.958625 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.958756 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.958886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.959211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.959370 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.959502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.959632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.959746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] trust.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.959862 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.959998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.960144 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.960274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.960632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.960719 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.960816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.960923 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.961022 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.961103 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.961175 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.961272 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.961372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.961457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.961531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.961611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.961684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.961755 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.connection = **** {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.961816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.961816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.962030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.962124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.962199 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.962271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.962361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.962486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.962618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.962749 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.962918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.963072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.963205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.963338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.963467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.963628 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.963761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.963888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.964015 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.964146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.964280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.964280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.964280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.964280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.964280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.964844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.964920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.964920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.965070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.965133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.965204 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.965275 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.965275 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=105315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 07 13:07:00.965407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.password = **** {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.965407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.965407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.965407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.965407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.965407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.965407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.965407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.965407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.966084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.966084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.966084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.966084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.966084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.966084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.966411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.966544 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.966818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.967097 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.967356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.967613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.967869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.968121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.968396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.968640 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.970031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.970031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.970031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.970031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.970031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.970399 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.970459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.970706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.971003 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.971288 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.971565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.971810 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.972055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.972329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.972613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.972893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.973142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.973450 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] policy.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.973724 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.973980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.974252 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.974535 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] receipt.caching = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.974916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] resource.caching = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] resource.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] revoke.caching = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] role.cache_time = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] role.caching = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] role.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] role.list_limit = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.981741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.982233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.982233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.982233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.982233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.982233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.982233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.982233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.982233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.982620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.982620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.982902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.983129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.983573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.983880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.984139 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.984394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.984647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.984937 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.985139 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.985466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.985739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.986000 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.986231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.986467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.986704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.986960 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.987240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.987495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.987738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.987977 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.988210 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] token.cache_time = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.988451 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] token.caching = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.988692 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.988968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] token.provider = fernet {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.989275 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.989512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.989765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.990009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.990259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.991069 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.992037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] trust.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.992292 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.992539 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.992781 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.993034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.993321 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.993409 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:00.993558 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.993651 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:00.993809 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.993898 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:00.994081 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.994348 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.connection = **** {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.994595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.994850 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.995113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.995401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.995645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.995888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.996134 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.996384 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.996635 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.996872 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.997116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.997357 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.997608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.997847 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.998095 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.998329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.998573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.998812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 13:07:00.999017 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=105316) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 07 13:07:01.002622 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.029210 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.029576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.029747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.033106 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.033347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.040128 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.060330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.060557 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.091646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.105661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.142627 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.142861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.152753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.152971 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.216483 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.216749 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.216977 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.217203 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.217427 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.217646 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.247320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.247546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.247766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.247997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.248221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.248446 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.248699 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.298590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.298831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.299071 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.299298 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.299517 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.299742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.313394 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.313646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.313834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.314059 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.314284 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.314497 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.314712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.328196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.343457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.343626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.343826 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.344054 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.344279 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.344496 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.344715 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=105315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.353895 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.356852 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.357085 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.357311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.357535 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.357754 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.357970 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.358199 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=105316) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 13:07:01.362291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.362503 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 13:07:01.362856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.363096 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 13:07:01.363376 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.363642 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.363908 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.364174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.369031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.369243 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 13:07:01.369617 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.369823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 13:07:01.370106 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.370370 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.370634 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.370898 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.373913 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.374201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.374481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.374772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.380583 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.380913 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.381682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.382325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.382566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.385302 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.385590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.385882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.386260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.386594 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.386819 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.387132 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.387414 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.387694 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.387978 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.388256 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.389742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.392758 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.393047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.393339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.393626 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.393918 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.394210 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.394543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.394822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.395122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.395403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.395790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.411742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.413559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.413852 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.418729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.419045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.419343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.420710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.422276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.422559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.425959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.426269 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.427474 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.427780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.428763 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.433119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.433433 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.433732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.434032 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.434311 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.434599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.434878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.435204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.435958 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.436262 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.440504 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.440804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.441216 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.441511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.441793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.442078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.442359 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.442632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.451807 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.452105 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.456029 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.466246 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.466246 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.466429 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.466429 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.466666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.466745 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.467297 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.468023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.468327 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.470547 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.470855 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.471172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.471468 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.471752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.472053 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.472647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.476582 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.480581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.480581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.480581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.480843 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.481250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.481567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.488492 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.488810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.489090 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.489361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.497162 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.498782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.500356 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.500478 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.500642 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.500878 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.501041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.501311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.503326 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.503636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.503926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.504221 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.504633 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.509672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.511276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.512816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.513101 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.515789 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.516082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.516371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.516681 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.516900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.517089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.517224 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.517358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.517553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.517957 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.518242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.518535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.529112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.529431 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.529726 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.530020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.530421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.530707 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.531027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.534186 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.537450 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.540446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.541916 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.543427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.543715 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.544156 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.547658 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.550629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.550762 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.552261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.554933 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.555796 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.556213 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.556416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.556524 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.556813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.557966 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.558264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.558682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.565044 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.565302 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.565442 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.565642 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.566803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.571232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.571695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.571988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.572266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.573399 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=105315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.573718 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.574010 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.574307 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.574593 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.574883 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.575193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.575472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.575752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.576037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.576335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.576602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.576875 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.577160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=105315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.580593 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.580903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.581227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.590416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=105316) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 13:07:01.590717 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.591032 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.591329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.591644 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.591945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.592225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.592512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.592797 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.593084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.593374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.593698 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.593920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.594183 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=105316) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 13:07:01.613841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55affe43d9c0 pid: 105315 (default app) Sep 07 13:07:01.618648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9392d5-cde9-48d9-b685-aa1d2092fbd9 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:01.618935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9392d5-cde9-48d9-b685-aa1d2092fbd9 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:01.619250 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9392d5-cde9-48d9-b685-aa1d2092fbd9 None None] PATH_INFO: `/v3` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:01.623626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1/1] 10.222.0.62 () {50 vars in 791 bytes} [Wed Sep 7 13:07:01 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:01.630959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55affe43d9c0 pid: 105316 (default app) Sep 07 13:07:09.922130 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46d8c3-7569-4913-965d-852dbdda009e None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:09.922652 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46d8c3-7569-4913-965d-852dbdda009e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:09.922652 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46d8c3-7569-4913-965d-852dbdda009e None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:09.924072 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 2/2] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:07:09.932084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391325f-6bb6-42db-a2d9-da309abf8e18 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:09.932611 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391325f-6bb6-42db-a2d9-da309abf8e18 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:09.932611 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391325f-6bb6-42db-a2d9-da309abf8e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:10.005500 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: INFO dbcounter [None req-f391325f-6bb6-42db-a2d9-da309abf8e18 None None] Registered counter for database keystone Sep 07 13:07:10.028102 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG oslo_db.sqlalchemy.engines [None req-f391325f-6bb6-42db-a2d9-da309abf8e18 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105316) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 07 13:07:10.034560 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writer thread running {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 07 13:07:10.219562 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG passlib.handlers.bcrypt [None req-f391325f-6bb6-42db-a2d9-da309abf8e18 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=105316) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 07 13:07:10.219875 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG passlib.handlers.bcrypt [None req-f391325f-6bb6-42db-a2d9-da309abf8e18 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=105316) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 07 13:07:10.279950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-f391325f-6bb6-42db-a2d9-da309abf8e18 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:10.281582 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f391325f-6bb6-42db-a2d9-da309abf8e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:10.423736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1/3] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 495 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 07 13:07:10.924547 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46d8c3-7569-4913-965d-852dbdda009e None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:11.001026 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: INFO dbcounter [None req-7e46d8c3-7569-4913-965d-852dbdda009e None None] Registered counter for database keystone Sep 07 13:07:11.020877 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG oslo_db.sqlalchemy.engines [None req-7e46d8c3-7569-4913-965d-852dbdda009e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105315) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 07 13:07:11.027729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writer thread running {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 07 13:07:11.147546 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46d8c3-7569-4913-965d-852dbdda009e None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:11.152013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f00c78-a8f8-4769-b191-10b75f38d7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:11.152650 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f00c78-a8f8-4769-b191-10b75f38d7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:11.152931 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f00c78-a8f8-4769-b191-10b75f38d7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:11.153207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f00c78-a8f8-4769-b191-10b75f38d7c7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:11.163110 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f00c78-a8f8-4769-b191-10b75f38d7c7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:11.216419 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: WARNING keystone.common.rbac_enforcer.enforcer [None req-11f00c78-a8f8-4769-b191-10b75f38d7c7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 07 13:07:11.216683 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f00c78-a8f8-4769-b191-10b75f38d7c7 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:11.227936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 3/4] 10.222.0.62 () {58 vars in 1324 bytes} [Wed Sep 7 13:07:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 13:07:11.234224 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391325f-6bb6-42db-a2d9-da309abf8e18 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:11.250844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391325f-6bb6-42db-a2d9-da309abf8e18 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:11.256566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98e2ce-8592-4391-8087-74137b8b2e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:11.258321 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98e2ce-8592-4391-8087-74137b8b2e72 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:11.258612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98e2ce-8592-4391-8087-74137b8b2e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:11.258894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98e2ce-8592-4391-8087-74137b8b2e72 admin admin] PATH_INFO: `/v3/projects/0ee134892b8643809999afc672e86085` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:11.260795 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98e2ce-8592-4391-8087-74137b8b2e72 admin admin] RBAC: Authorizing `identity:get_project(project_id=0ee134892b8643809999afc672e86085)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:11.316257 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0f98e2ce-8592-4391-8087-74137b8b2e72 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 07 13:07:11.316601 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98e2ce-8592-4391-8087-74137b8b2e72 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:11.322714 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 2/5] 10.222.0.62 () {56 vars in 1207 bytes} [Wed Sep 7 13:07:11 2022] GET /identity/v3/projects/0ee134892b8643809999afc672e86085 => generated 344 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:13.576192 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba36e7-e65b-4873-81f9-acf838fd22ea None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:13.576745 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba36e7-e65b-4873-81f9-acf838fd22ea None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:13.576745 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba36e7-e65b-4873-81f9-acf838fd22ea None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:13.580968 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 4/6] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:13.587726 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e16ff-21e4-4c06-b119-5106edcf2df4 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:13.588226 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e16ff-21e4-4c06-b119-5106edcf2df4 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:13.588311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e16ff-21e4-4c06-b119-5106edcf2df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:13.608519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-e83e16ff-21e4-4c06-b119-5106edcf2df4 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:13.610043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e83e16ff-21e4-4c06-b119-5106edcf2df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:13.618921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 3/7] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 07 13:07:13.888766 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba36e7-e65b-4873-81f9-acf838fd22ea None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:13.903276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba36e7-e65b-4873-81f9-acf838fd22ea None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:13.907834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20965a60-fb60-4307-8aba-e33763560ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:13.908471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20965a60-fb60-4307-8aba-e33763560ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:13.908747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20965a60-fb60-4307-8aba-e33763560ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:13.909017 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20965a60-fb60-4307-8aba-e33763560ce8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:13.919268 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20965a60-fb60-4307-8aba-e33763560ce8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:13.921326 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20965a60-fb60-4307-8aba-e33763560ce8 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:13.932573 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 5/8] 10.222.0.62 () {58 vars in 1324 bytes} [Wed Sep 7 13:07:13 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 13:07:13.938481 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e16ff-21e4-4c06-b119-5106edcf2df4 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:13.949008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e16ff-21e4-4c06-b119-5106edcf2df4 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:13.953853 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05b5f8-998f-4ed5-8980-4eae65ed1320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:13.954519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05b5f8-998f-4ed5-8980-4eae65ed1320 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:13.954804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05b5f8-998f-4ed5-8980-4eae65ed1320 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:13.955126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05b5f8-998f-4ed5-8980-4eae65ed1320 admin admin] PATH_INFO: `/v3/users/7a8e484a77ce40f880d21a33a930f0be` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:13.956543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05b5f8-998f-4ed5-8980-4eae65ed1320 admin admin] RBAC: Authorizing `identity:get_user(user_id=7a8e484a77ce40f880d21a33a930f0be)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:13.958444 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05b5f8-998f-4ed5-8980-4eae65ed1320 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:13.961206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 4/9] 10.222.0.62 () {56 vars in 1198 bytes} [Wed Sep 7 13:07:13 2022] GET /identity/v3/users/7a8e484a77ce40f880d21a33a930f0be => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:16.369867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcadab5-ab9e-4ebe-9fd8-c776c939cf26 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:16.370536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcadab5-ab9e-4ebe-9fd8-c776c939cf26 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:16.370717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcadab5-ab9e-4ebe-9fd8-c776c939cf26 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:16.372269 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 6/10] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:07:16.378191 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425fee9-fc0d-41d2-86a9-d9aa296874ad None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:16.378735 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425fee9-fc0d-41d2-86a9-d9aa296874ad None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:16.378735 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425fee9-fc0d-41d2-86a9-d9aa296874ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:16.398027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-2425fee9-fc0d-41d2-86a9-d9aa296874ad None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:16.399301 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2425fee9-fc0d-41d2-86a9-d9aa296874ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:16.406219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68633d42-c5fb-4885-a71b-5f231cdd68cb None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:16.406615 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68633d42-c5fb-4885-a71b-5f231cdd68cb None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:16.406994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68633d42-c5fb-4885-a71b-5f231cdd68cb None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:16.408540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 7/11] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:07:16.415677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a50da5-7351-4c36-b3e6-60213015dce4 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:16.416103 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a50da5-7351-4c36-b3e6-60213015dce4 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:16.416484 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a50da5-7351-4c36-b3e6-60213015dce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:16.464214 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG passlib.handlers.bcrypt [None req-37a50da5-7351-4c36-b3e6-60213015dce4 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=105315) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 07 13:07:16.464599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG passlib.handlers.bcrypt [None req-37a50da5-7351-4c36-b3e6-60213015dce4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=105315) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 07 13:07:16.496818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 5/12] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:16.514530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-37a50da5-7351-4c36-b3e6-60213015dce4 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:16.516296 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-37a50da5-7351-4c36-b3e6-60213015dce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:16.622148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 8/13] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:07:16.780772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425fee9-fc0d-41d2-86a9-d9aa296874ad None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:16.812572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:16.894585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a50da5-7351-4c36-b3e6-60213015dce4 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:16.901526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425fee9-fc0d-41d2-86a9-d9aa296874ad None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:16.939270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851fbe9-0345-4211-8ad0-1488a4016d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:16.939918 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851fbe9-0345-4211-8ad0-1488a4016d0d None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:16.940199 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851fbe9-0345-4211-8ad0-1488a4016d0d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:16.940481 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851fbe9-0345-4211-8ad0-1488a4016d0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.026107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851fbe9-0345-4211-8ad0-1488a4016d0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:17.027652 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851fbe9-0345-4211-8ad0-1488a4016d0d None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:17.053013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:17.053206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a50da5-7351-4c36-b3e6-60213015dce4 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:17.056022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:17.095826 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd9c43-18d8-4700-ab2b-8d611abd0df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:17.097271 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd9c43-18d8-4700-ab2b-8d611abd0df3 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.097666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd9c43-18d8-4700-ab2b-8d611abd0df3 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.097973 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd9c43-18d8-4700-ab2b-8d611abd0df3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.112736 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd9c43-18d8-4700-ab2b-8d611abd0df3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:17.123078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-fbfd9c43-18d8-4700-ab2b-8d611abd0df3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:07:17.123078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:17.123078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:17.123078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:17.123078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:17.123078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:17.123078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:17.123078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:17.123078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:17.123078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:17.123078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:17.123078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:17.123078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:17.123078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:17.123078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:17.123078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:17.124207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:17.124207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:17.124207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:17.124207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:17.124207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:17.124207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:17.124207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:17.124207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:17.124207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:17.124207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:17.124207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:17.124207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:17.124207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:17.124207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:17.124207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:17.124207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:17.124974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:17.124974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:17.124974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:17.124974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:17.124974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:17.124974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:17.124974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:17.124974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:17.124974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:17.124974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:17.124974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:07:17.124974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:07:17.124974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 9/14] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:07:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:07:17.132953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd9c43-18d8-4700-ab2b-8d611abd0df3 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:17.150556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 6/15] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:07:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:07:17.157002 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851fbe9-0345-4211-8ad0-1488a4016d0d None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:17.261839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd9c43-18d8-4700-ab2b-8d611abd0df3 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:17.291382 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851fbe9-0345-4211-8ad0-1488a4016d0d None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:17.322221 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc4747-105c-4160-b87f-3e740669d0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:17.322957 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc4747-105c-4160-b87f-3e740669d0b5 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.323148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:17.323272 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc4747-105c-4160-b87f-3e740669d0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.323531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc4747-105c-4160-b87f-3e740669d0b5 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.324387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc4747-105c-4160-b87f-3e740669d0b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:17.324723 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc4747-105c-4160-b87f-3e740669d0b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:17.326326 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc4747-105c-4160-b87f-3e740669d0b5 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:17.339914 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82347ae0-5b6b-42bb-bcbb-99ea760291f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:17.340704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82347ae0-5b6b-42bb-bcbb-99ea760291f1 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.344884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82347ae0-5b6b-42bb-bcbb-99ea760291f1 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.344884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82347ae0-5b6b-42bb-bcbb-99ea760291f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.345087 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 10/16] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:07:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:17.348047 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc4747-105c-4160-b87f-3e740669d0b5 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:17.356488 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82347ae0-5b6b-42bb-bcbb-99ea760291f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:17.358167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82347ae0-5b6b-42bb-bcbb-99ea760291f1 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:17.379718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-82347ae0-5b6b-42bb-bcbb-99ea760291f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:07:17.379718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:17.379718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:17.379718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:17.379718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:17.379718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:17.379718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:17.379718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:17.379718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:17.379718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:17.379718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:17.379718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:17.379718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:07:17.379718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:07:17.379718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:07:17.379718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:07:17.382006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:17.382006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:17.382006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:17.382006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:17.382006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:17.382006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:17.382006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:17.382006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:17.382006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:17.382006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:17.382006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:17.382006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:17.382006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:17.382006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:17.382006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:17.382006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:17.382006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:07:17.382857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:07:17.382857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:07:17.382857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:07:17.383048 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:17.383421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 7/17] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:07:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:07:17.387390 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82347ae0-5b6b-42bb-bcbb-99ea760291f1 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:17.468138 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc4747-105c-4160-b87f-3e740669d0b5 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:17.516379 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82347ae0-5b6b-42bb-bcbb-99ea760291f1 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:17.519091 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0eef6-6ebf-4979-91fb-3538004bf12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:17.520120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0eef6-6ebf-4979-91fb-3538004bf12c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.520572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0eef6-6ebf-4979-91fb-3538004bf12c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.521724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0eef6-6ebf-4979-91fb-3538004bf12c None admin] PATH_INFO: `/v3/users/7a8e484a77ce40f880d21a33a930f0be` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.525027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0eef6-6ebf-4979-91fb-3538004bf12c None admin] RBAC: Authorizing `identity:get_user(user_id=7a8e484a77ce40f880d21a33a930f0be)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:17.527641 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0eef6-6ebf-4979-91fb-3538004bf12c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:17.530445 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 11/18] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:17 2022] GET /identity/v3/users/7a8e484a77ce40f880d21a33a930f0be => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:17.536187 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0eef6-6ebf-4979-91fb-3538004bf12c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:17.558430 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddebd88-76eb-4c02-8eb5-6dde673684b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:17.559084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddebd88-76eb-4c02-8eb5-6dde673684b7 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.559370 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddebd88-76eb-4c02-8eb5-6dde673684b7 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.559645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddebd88-76eb-4c02-8eb5-6dde673684b7 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.560296 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddebd88-76eb-4c02-8eb5-6dde673684b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:17.560601 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddebd88-76eb-4c02-8eb5-6dde673684b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:17.562012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddebd88-76eb-4c02-8eb5-6dde673684b7 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:17.581486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 8/19] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:17.653789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:17.661087 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0eef6-6ebf-4979-91fb-3538004bf12c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:17.704742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e2bee-4e0c-4e18-8087-1ade5b3f9b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:17.705404 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e2bee-4e0c-4e18-8087-1ade5b3f9b1d None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.705686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e2bee-4e0c-4e18-8087-1ade5b3f9b1d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.705963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e2bee-4e0c-4e18-8087-1ade5b3f9b1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.707354 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e2bee-4e0c-4e18-8087-1ade5b3f9b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:17.709581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e2bee-4e0c-4e18-8087-1ade5b3f9b1d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:17.712078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 12/20] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:07:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:17.717825 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddebd88-76eb-4c02-8eb5-6dde673684b7 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:17.756905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:17.836912 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddebd88-76eb-4c02-8eb5-6dde673684b7 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:17.880209 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97627aec-e4f9-42fe-9491-1d7319944d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:17.881535 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97627aec-e4f9-42fe-9491-1d7319944d3a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:17.881848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97627aec-e4f9-42fe-9491-1d7319944d3a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:17.882106 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97627aec-e4f9-42fe-9491-1d7319944d3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:17.883069 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97627aec-e4f9-42fe-9491-1d7319944d3a None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.domain.id=default, user.id=7a8e484a77ce40f880d21a33a930f0be) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:17.883238 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97627aec-e4f9-42fe-9491-1d7319944d3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:17.884738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97627aec-e4f9-42fe-9491-1d7319944d3a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:17.892670 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 9/21] 10.222.0.62 () {56 vars in 1322 bytes} [Wed Sep 7 13:07:17 2022] GET /identity/v3/role_assignments?user.id=7a8e484a77ce40f880d21a33a930f0be&scope.domain.id=default&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 233 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:20.436900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02020ada-d045-480c-a65e-5ff02854cf3f None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:20.437458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02020ada-d045-480c-a65e-5ff02854cf3f None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:20.437458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02020ada-d045-480c-a65e-5ff02854cf3f None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:20.439239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 13/22] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:20.455381 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a1a5b-2ad5-4e65-a936-fb4def899481 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:20.455961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a1a5b-2ad5-4e65-a936-fb4def899481 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:20.455961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a1a5b-2ad5-4e65-a936-fb4def899481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:20.492369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-b59a1a5b-2ad5-4e65-a936-fb4def899481 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:20.493675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b59a1a5b-2ad5-4e65-a936-fb4def899481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:20.540193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c55a63-8d42-4c6f-b2ea-9a5b4f242dba None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:20.540856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c55a63-8d42-4c6f-b2ea-9a5b4f242dba None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:20.541323 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c55a63-8d42-4c6f-b2ea-9a5b4f242dba None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:20.542934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 14/23] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:07:20.549376 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458a025-e35e-4358-98f8-97a28e67440a None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:20.549975 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458a025-e35e-4358-98f8-97a28e67440a None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:20.550466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458a025-e35e-4358-98f8-97a28e67440a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:20.570684 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-c458a025-e35e-4358-98f8-97a28e67440a None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:20.572271 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c458a025-e35e-4358-98f8-97a28e67440a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:20.622392 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 10/24] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:20.687895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 15/25] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:07:21.016915 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a1a5b-2ad5-4e65-a936-fb4def899481 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:21.034714 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f035d-c317-412c-b0d4-8cb96bbeb075 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:21.035350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f035d-c317-412c-b0d4-8cb96bbeb075 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:21.035834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f035d-c317-412c-b0d4-8cb96bbeb075 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:21.038012 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 16/26] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:07:21.044215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e244841-3a3c-4b75-acb2-e49900c05f75 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:21.044802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e244841-3a3c-4b75-acb2-e49900c05f75 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:21.045263 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e244841-3a3c-4b75-acb2-e49900c05f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:21.071756 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:21.077270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-1e244841-3a3c-4b75-acb2-e49900c05f75 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:21.078758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1e244841-3a3c-4b75-acb2-e49900c05f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:21.211828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 17/27] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:07:21.258119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19087a92-ab98-412d-8752-4398664d3ec1 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:21.258724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19087a92-ab98-412d-8752-4398664d3ec1 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:21.259243 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19087a92-ab98-412d-8752-4398664d3ec1 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:21.259964 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a1a5b-2ad5-4e65-a936-fb4def899481 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:21.261141 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 18/28] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:07:21.269206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e21de-5391-4ea7-919c-ce2e20c0356c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:21.269741 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e21de-5391-4ea7-919c-ce2e20c0356c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:21.271413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e21de-5391-4ea7-919c-ce2e20c0356c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:21.305372 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b53e21de-5391-4ea7-919c-ce2e20c0356c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:21.307145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b53e21de-5391-4ea7-919c-ce2e20c0356c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:21.336741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d3ea4-c247-404d-a76a-bf7a30a372ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:21.337832 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3ea4-c247-404d-a76a-bf7a30a372ac None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:21.338309 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3ea4-c247-404d-a76a-bf7a30a372ac None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:21.338787 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3ea4-c247-404d-a76a-bf7a30a372ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:21.351984 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:21.376119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d3ea4-c247-404d-a76a-bf7a30a372ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:21.378935 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d3ea4-c247-404d-a76a-bf7a30a372ac None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:21.413886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-a26d3ea4-c247-404d-a76a-bf7a30a372ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:07:21.413886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:21.413886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:21.413886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:21.413886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:21.413886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:21.413886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:21.413886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:21.413886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:21.413886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:21.413886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:21.413886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:21.413886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:07:21.413886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:07:21.413886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:07:21.413886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:07:21.414816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:21.414816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:21.414816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:21.414816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:21.414816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:21.414816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:21.414816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:21.414816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:21.414816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:21.414816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:21.414816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:21.414816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:21.414816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:21.414816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:21.414816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:21.414816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:21.414816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:07:21.415992 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:07:21.415992 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:07:21.415992 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:07:21.416150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 11/29] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:07:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:07:21.417868 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d3ea4-c247-404d-a76a-bf7a30a372ac None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:21.493174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 19/30] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 07 13:07:21.494786 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e21de-5391-4ea7-919c-ce2e20c0356c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:21.627637 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:21.673238 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d3ea4-c247-404d-a76a-bf7a30a372ac None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:21.728881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e21de-5391-4ea7-919c-ce2e20c0356c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:21.741216 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d3511-efe9-4d75-9ebd-019bc3df1add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:21.741844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d3511-efe9-4d75-9ebd-019bc3df1add None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:21.742126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d3511-efe9-4d75-9ebd-019bc3df1add None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:21.742396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d3511-efe9-4d75-9ebd-019bc3df1add None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:21.743226 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d3511-efe9-4d75-9ebd-019bc3df1add None admin] RBAC: Authorizing `identity:create_role()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:21.744773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d3511-efe9-4d75-9ebd-019bc3df1add None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:21.782673 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e16755-f881-4bd7-9ae4-fba241a1ee23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:21.783363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e16755-f881-4bd7-9ae4-fba241a1ee23 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:21.783645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e16755-f881-4bd7-9ae4-fba241a1ee23 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:21.783968 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e16755-f881-4bd7-9ae4-fba241a1ee23 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:21.784633 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e16755-f881-4bd7-9ae4-fba241a1ee23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:21.788188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e16755-f881-4bd7-9ae4-fba241a1ee23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:21.788188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e16755-f881-4bd7-9ae4-fba241a1ee23 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:21.813453 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 20/31] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:21.814368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e16755-f881-4bd7-9ae4-fba241a1ee23 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:21.832770 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:21.978324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e16755-f881-4bd7-9ae4-fba241a1ee23 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:22.050917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca73238-c7e0-4bce-9645-86354411a163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:22.052437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca73238-c7e0-4bce-9645-86354411a163 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:22.052437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca73238-c7e0-4bce-9645-86354411a163 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:22.053183 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca73238-c7e0-4bce-9645-86354411a163 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:22.075608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca73238-c7e0-4bce-9645-86354411a163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:22.087359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-4ca73238-c7e0-4bce-9645-86354411a163 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:07:22.087359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:22.087359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:22.087359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:22.087359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:22.087359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:22.087359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:22.087359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:22.087359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:22.087359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:22.087359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:22.087359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:22.087359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:22.087359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:22.087359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:22.087359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:22.088273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:22.088273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:22.088273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:22.088273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:22.088273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:22.088273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:22.088273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:22.088273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:22.088273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:22.088273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:22.088273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:22.088273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:22.088273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:22.088273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:22.088273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:22.088273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:22.089022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:22.089022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:22.089022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:22.089022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:22.089022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:22.089022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:22.089022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:22.089022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:22.089022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:22.089022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:22.089022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:07:22.089022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:07:22.089590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 21/32] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:07:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:07:22.093380 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca73238-c7e0-4bce-9645-86354411a163 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:22.097990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 12/33] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Sep 7 13:07:21 2022] POST /identity/v3/roles => generated 219 bytes in 681 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:22.099935 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d3511-efe9-4d75-9ebd-019bc3df1add None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:22.252767 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:22.325904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca73238-c7e0-4bce-9645-86354411a163 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:22.388864 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d3511-efe9-4d75-9ebd-019bc3df1add None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:22.406330 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0814a02a-2a8e-4926-ab14-3a0abe360edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:22.407402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814a02a-2a8e-4926-ab14-3a0abe360edc None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:22.407857 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814a02a-2a8e-4926-ab14-3a0abe360edc None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:22.409190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0814a02a-2a8e-4926-ab14-3a0abe360edc None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:22.410498 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814a02a-2a8e-4926-ab14-3a0abe360edc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:22.414920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0814a02a-2a8e-4926-ab14-3a0abe360edc None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:22.485378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5cd3b-c548-4785-8745-f9e6f0c8a293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:22.486561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5cd3b-c548-4785-8745-f9e6f0c8a293 None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:22.487035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5cd3b-c548-4785-8745-f9e6f0c8a293 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:22.487618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5cd3b-c548-4785-8745-f9e6f0c8a293 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:22.488772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5cd3b-c548-4785-8745-f9e6f0c8a293 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:22.492054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5cd3b-c548-4785-8745-f9e6f0c8a293 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:22.703010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:22.710148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 13/34] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Sep 7 13:07:22 2022] POST /identity/v3/roles => generated 225 bytes in 610 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:22.710148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5cd3b-c548-4785-8745-f9e6f0c8a293 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:22.775013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 22/35] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 7 13:07:22 2022] POST /identity/v3/projects => generated 301 bytes in 676 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:22.775013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf15f75-3adb-4f45-8b5a-2f6b39d02b4f None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:22.775013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf15f75-3adb-4f45-8b5a-2f6b39d02b4f None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:22.775013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf15f75-3adb-4f45-8b5a-2f6b39d02b4f None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:22.775013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 23/36] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:22.775013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9e360-f7f5-4f5d-b65e-fe015130c2c1 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:22.775013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9e360-f7f5-4f5d-b65e-fe015130c2c1 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:22.776273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9e360-f7f5-4f5d-b65e-fe015130c2c1 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:22.776273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 24/37] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:22.782074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d48d3a-f0b1-47ef-84a9-85a6ade3de19 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:22.782508 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d48d3a-f0b1-47ef-84a9-85a6ade3de19 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:22.783023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d48d3a-f0b1-47ef-84a9-85a6ade3de19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:22.816095 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-75d48d3a-f0b1-47ef-84a9-85a6ade3de19 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:22.818581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-75d48d3a-f0b1-47ef-84a9-85a6ade3de19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:22.865992 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5cd3b-c548-4785-8745-f9e6f0c8a293 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:22.917934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27102a-afba-4e8a-bdeb-884c7a0ff6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:22.918639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27102a-afba-4e8a-bdeb-884c7a0ff6c4 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:22.918939 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27102a-afba-4e8a-bdeb-884c7a0ff6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:22.919265 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27102a-afba-4e8a-bdeb-884c7a0ff6c4 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:22.920005 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27102a-afba-4e8a-bdeb-884c7a0ff6c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:22.920318 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27102a-afba-4e8a-bdeb-884c7a0ff6c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:22.921967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27102a-afba-4e8a-bdeb-884c7a0ff6c4 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:22.938370 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 14/38] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:07:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:22.940801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f324f49-1bf5-4a98-9254-c48a0d59c2fb None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:22.941121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f324f49-1bf5-4a98-9254-c48a0d59c2fb None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:22.941479 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f324f49-1bf5-4a98-9254-c48a0d59c2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:22.961818 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 25/39] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:22.962895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d48d3a-f0b1-47ef-84a9-85a6ade3de19 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:22.964278 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-1f324f49-1bf5-4a98-9254-c48a0d59c2fb None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:22.965899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-1f324f49-1bf5-4a98-9254-c48a0d59c2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:23.053871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:23.119561 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d48d3a-f0b1-47ef-84a9-85a6ade3de19 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:23.122417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:23.124532 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 15/40] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:23.165850 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58472140-44a7-4bdf-bdee-d5b6491cf07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:23.166513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58472140-44a7-4bdf-bdee-d5b6491cf07e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:23.166788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58472140-44a7-4bdf-bdee-d5b6491cf07e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:23.167095 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58472140-44a7-4bdf-bdee-d5b6491cf07e None admin] PATH_INFO: `/v3/users/7a8e484a77ce40f880d21a33a930f0be` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:23.168741 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58472140-44a7-4bdf-bdee-d5b6491cf07e None admin] RBAC: Authorizing `identity:get_user(user_id=7a8e484a77ce40f880d21a33a930f0be)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:23.170222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58472140-44a7-4bdf-bdee-d5b6491cf07e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:23.171975 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 26/41] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:22 2022] GET /identity/v3/users/7a8e484a77ce40f880d21a33a930f0be => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:23.177787 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f324f49-1bf5-4a98-9254-c48a0d59c2fb None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:23.297619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58472140-44a7-4bdf-bdee-d5b6491cf07e None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:23.306055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f324f49-1bf5-4a98-9254-c48a0d59c2fb None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:23.357530 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab39d4-8fbe-4125-a291-d2398ae43711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:23.358583 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab39d4-8fbe-4125-a291-d2398ae43711 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:23.359403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab39d4-8fbe-4125-a291-d2398ae43711 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:23.359578 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab39d4-8fbe-4125-a291-d2398ae43711 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:23.361467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab39d4-8fbe-4125-a291-d2398ae43711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:23.364147 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab39d4-8fbe-4125-a291-d2398ae43711 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:23.371718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 16/42] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:07:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:23.373156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab39d4-8fbe-4125-a291-d2398ae43711 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:23.430509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58472140-44a7-4bdf-bdee-d5b6491cf07e None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:23.441408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:23.460880 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:23.472805 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe539189-c6d6-4170-b6ab-dfe2d4171dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:23.473455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe539189-c6d6-4170-b6ab-dfe2d4171dae None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:23.473763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe539189-c6d6-4170-b6ab-dfe2d4171dae None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:23.475419 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe539189-c6d6-4170-b6ab-dfe2d4171dae None admin] PATH_INFO: `/v3/domains/default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:23.475769 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe539189-c6d6-4170-b6ab-dfe2d4171dae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:23.477252 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe539189-c6d6-4170-b6ab-dfe2d4171dae None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:23.479257 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 27/43] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:07:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:23.480063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe539189-c6d6-4170-b6ab-dfe2d4171dae None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:23.489839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab39d4-8fbe-4125-a291-d2398ae43711 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:23.530566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27b1ce-2b84-45e1-8229-752b78f0efbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:23.531276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27b1ce-2b84-45e1-8229-752b78f0efbd None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:23.531562 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27b1ce-2b84-45e1-8229-752b78f0efbd None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:23.531847 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27b1ce-2b84-45e1-8229-752b78f0efbd None admin] PATH_INFO: `/v3/domains/default/users/7a8e484a77ce40f880d21a33a930f0be/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:23.534041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27b1ce-2b84-45e1-8229-752b78f0efbd None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7a8e484a77ce40f880d21a33a930f0be, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:23.535658 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27b1ce-2b84-45e1-8229-752b78f0efbd None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:23.624592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe539189-c6d6-4170-b6ab-dfe2d4171dae None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:23.655654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 17/44] 10.222.0.62 () {58 vars in 1361 bytes} [Wed Sep 7 13:07:23 2022] PUT /identity/v3/domains/default/users/7a8e484a77ce40f880d21a33a930f0be/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:07:23.661904 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27b1ce-2b84-45e1-8229-752b78f0efbd None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:23.675294 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff243c8-767a-4f05-8b6b-5f33f0b2bcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:23.676745 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff243c8-767a-4f05-8b6b-5f33f0b2bcf1 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:23.677834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff243c8-767a-4f05-8b6b-5f33f0b2bcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:23.678192 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff243c8-767a-4f05-8b6b-5f33f0b2bcf1 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:23.679879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff243c8-767a-4f05-8b6b-5f33f0b2bcf1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:23.680465 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff243c8-767a-4f05-8b6b-5f33f0b2bcf1 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:23.772411 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 28/45] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Sep 7 13:07:23 2022] POST /identity/v3/roles => generated 223 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:23.913757 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27b1ce-2b84-45e1-8229-752b78f0efbd None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:23.939783 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:23.957024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ac949-42e6-408d-ace8-3b12fcc08f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:23.957641 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ac949-42e6-408d-ace8-3b12fcc08f87 None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:23.957927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ac949-42e6-408d-ace8-3b12fcc08f87 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:23.958299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ac949-42e6-408d-ace8-3b12fcc08f87 None admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:23.958998 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ac949-42e6-408d-ace8-3b12fcc08f87 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:23.960774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ac949-42e6-408d-ace8-3b12fcc08f87 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:24.092712 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 18/46] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 7 13:07:23 2022] POST /identity/v3/projects => generated 312 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:25.979213 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7de986-2757-4ac8-ae5b-ee7081e50a77 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:25.979775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7de986-2757-4ac8-ae5b-ee7081e50a77 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:25.980170 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7de986-2757-4ac8-ae5b-ee7081e50a77 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:25.982695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 29/47] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:25.989012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfa050-31e1-4310-a1ec-c79c8b7a191d None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:25.990424 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfa050-31e1-4310-a1ec-c79c8b7a191d None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:25.990571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfa050-31e1-4310-a1ec-c79c8b7a191d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:26.014585 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-b2bfa050-31e1-4310-a1ec-c79c8b7a191d None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:26.016674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b2bfa050-31e1-4310-a1ec-c79c8b7a191d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:26.129365 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 19/48] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:26.420226 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0390be7-a071-4e9c-8f72-77f59cb50fed None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:26.420775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0390be7-a071-4e9c-8f72-77f59cb50fed None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:26.421156 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0390be7-a071-4e9c-8f72-77f59cb50fed None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:26.423166 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 30/49] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:26.425476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfa050-31e1-4310-a1ec-c79c8b7a191d None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:26.429031 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb18eb-8afa-445f-9623-98c0d45eb7e3 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:26.429324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb18eb-8afa-445f-9623-98c0d45eb7e3 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:26.429605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb18eb-8afa-445f-9623-98c0d45eb7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:26.449895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-79fb18eb-8afa-445f-9623-98c0d45eb7e3 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:26.451167 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-79fb18eb-8afa-445f-9623-98c0d45eb7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:26.518922 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:26.556454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 31/50] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:07:26.580013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfa050-31e1-4310-a1ec-c79c8b7a191d None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:26.582841 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:26.628883 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a099f-5acf-458f-a645-e26e2d4e3a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:26.629534 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a099f-5acf-458f-a645-e26e2d4e3a41 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:26.629816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a099f-5acf-458f-a645-e26e2d4e3a41 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:26.630612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a099f-5acf-458f-a645-e26e2d4e3a41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:26.645073 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a099f-5acf-458f-a645-e26e2d4e3a41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:26.652676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-a59a099f-5acf-458f-a645-e26e2d4e3a41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:07:26.652676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:26.652676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:26.652676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:26.652676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:26.652676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:26.652676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:26.652676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:26.652676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:26.652676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:26.652676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:26.652676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:26.652676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:26.652676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:26.652676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:26.652676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:26.654361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:26.654361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:26.654361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:26.654361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:26.654361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:26.654361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:26.654361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:26.654361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:26.654361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:26.654361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:26.654361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:26.654361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:26.654361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:26.654361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:26.654361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:26.654361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:26.655154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:26.655154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:26.655154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:26.655154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:26.655154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:26.655154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:26.655154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:26.655154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:26.655154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:26.655154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:26.655154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:07:26.655154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:07:26.655154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 20/51] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:07:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:07:26.660715 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb18eb-8afa-445f-9623-98c0d45eb7e3 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:26.787493 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb18eb-8afa-445f-9623-98c0d45eb7e3 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:26.830515 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006c9b6-016b-44be-9a58-02c123fb6695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:26.831465 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006c9b6-016b-44be-9a58-02c123fb6695 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:26.831870 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006c9b6-016b-44be-9a58-02c123fb6695 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:26.832277 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006c9b6-016b-44be-9a58-02c123fb6695 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:26.833961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006c9b6-016b-44be-9a58-02c123fb6695 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:26.834412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006c9b6-016b-44be-9a58-02c123fb6695 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:26.836898 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006c9b6-016b-44be-9a58-02c123fb6695 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:26.851180 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 32/52] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:07:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:26.858121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a099f-5acf-458f-a645-e26e2d4e3a41 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:26.878592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006c9b6-016b-44be-9a58-02c123fb6695 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:26.939026 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:26.994419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a099f-5acf-458f-a645-e26e2d4e3a41 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:27.006567 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006c9b6-016b-44be-9a58-02c123fb6695 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:27.034401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d0082-e7fa-492f-a908-e2ab6b2b6f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:27.035092 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d0082-e7fa-492f-a908-e2ab6b2b6f43 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:27.035374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d0082-e7fa-492f-a908-e2ab6b2b6f43 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:27.035651 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d0082-e7fa-492f-a908-e2ab6b2b6f43 None admin] PATH_INFO: `/v3/users/7a8e484a77ce40f880d21a33a930f0be` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:27.037466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d0082-e7fa-492f-a908-e2ab6b2b6f43 None admin] RBAC: Authorizing `identity:get_user(user_id=7a8e484a77ce40f880d21a33a930f0be)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:27.039032 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d0082-e7fa-492f-a908-e2ab6b2b6f43 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:27.040829 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 21/53] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:26 2022] GET /identity/v3/users/7a8e484a77ce40f880d21a33a930f0be => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:27.046085 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d0082-e7fa-492f-a908-e2ab6b2b6f43 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:27.056735 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94cf41-d71c-4de4-b32c-96485c042492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:27.057510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94cf41-d71c-4de4-b32c-96485c042492 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:27.057510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94cf41-d71c-4de4-b32c-96485c042492 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:27.057777 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94cf41-d71c-4de4-b32c-96485c042492 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:27.058959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc94cf41-d71c-4de4-b32c-96485c042492 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:27.060469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc94cf41-d71c-4de4-b32c-96485c042492 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:27.062100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 33/54] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:07:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:27.067997 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94cf41-d71c-4de4-b32c-96485c042492 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:27.079709 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:27.171100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d0082-e7fa-492f-a908-e2ab6b2b6f43 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:27.220485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644d667-b2fc-4db1-8674-8bdf7ac36ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:27.221183 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644d667-b2fc-4db1-8674-8bdf7ac36ca2 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:27.221464 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644d667-b2fc-4db1-8674-8bdf7ac36ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:27.221740 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644d667-b2fc-4db1-8674-8bdf7ac36ca2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:27.223126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644d667-b2fc-4db1-8674-8bdf7ac36ca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:27.225842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644d667-b2fc-4db1-8674-8bdf7ac36ca2 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:27.227618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 22/55] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:07:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:27.235078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644d667-b2fc-4db1-8674-8bdf7ac36ca2 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:27.319910 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94cf41-d71c-4de4-b32c-96485c042492 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:27.332364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:27.359864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d5f32-f8ed-44aa-a805-c401f14bc354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:27.360625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d5f32-f8ed-44aa-a805-c401f14bc354 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:27.360992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d5f32-f8ed-44aa-a805-c401f14bc354 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:27.361357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d5f32-f8ed-44aa-a805-c401f14bc354 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:27.362158 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d5f32-f8ed-44aa-a805-c401f14bc354 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:27.363047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:27.363719 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d5f32-f8ed-44aa-a805-c401f14bc354 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:27.367836 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644d667-b2fc-4db1-8674-8bdf7ac36ca2 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:27.411985 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee332da-8d4b-4dff-b0ec-29a316f2be90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:27.412639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee332da-8d4b-4dff-b0ec-29a316f2be90 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:27.412926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee332da-8d4b-4dff-b0ec-29a316f2be90 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:27.413199 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee332da-8d4b-4dff-b0ec-29a316f2be90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:27.413936 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee332da-8d4b-4dff-b0ec-29a316f2be90 None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.domain.id=default, user.id=7a8e484a77ce40f880d21a33a930f0be) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:27.414242 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee332da-8d4b-4dff-b0ec-29a316f2be90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:27.415720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee332da-8d4b-4dff-b0ec-29a316f2be90 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:27.422102 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 23/56] 10.222.0.62 () {56 vars in 1322 bytes} [Wed Sep 7 13:07:27 2022] GET /identity/v3/role_assignments?user.id=7a8e484a77ce40f880d21a33a930f0be&scope.domain.id=default&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 530 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:27.542523 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 34/57] 10.222.0.62 () {60 vars in 1140 bytes} [Wed Sep 7 13:07:27 2022] POST /identity/v3/projects => generated 298 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:07:30.036976 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17be0c-39b8-4a13-acbd-8fa10e27944b None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:30.037712 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17be0c-39b8-4a13-acbd-8fa10e27944b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:30.037879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17be0c-39b8-4a13-acbd-8fa10e27944b None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:30.040605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 24/58] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:30.049319 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1eb224-c020-433a-8030-25df31ed8217 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:30.049861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1eb224-c020-433a-8030-25df31ed8217 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:30.049955 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1eb224-c020-433a-8030-25df31ed8217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:30.080052 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-4d1eb224-c020-433a-8030-25df31ed8217 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:30.082034 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-4d1eb224-c020-433a-8030-25df31ed8217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:30.182999 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 35/59] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:30.693207 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17be0c-39b8-4a13-acbd-8fa10e27944b None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:30.828740 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f17be0c-39b8-4a13-acbd-8fa10e27944b None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:30.865020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50c117-19ed-4dfd-810b-905bdcdc0a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:30.865695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50c117-19ed-4dfd-810b-905bdcdc0a1a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:30.865979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50c117-19ed-4dfd-810b-905bdcdc0a1a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:30.866256 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50c117-19ed-4dfd-810b-905bdcdc0a1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:30.867468 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50c117-19ed-4dfd-810b-905bdcdc0a1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:30.868961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50c117-19ed-4dfd-810b-905bdcdc0a1a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:30.870581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 25/60] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:07:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:30.880516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1eb224-c020-433a-8030-25df31ed8217 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:30.992788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:31.015377 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1eb224-c020-433a-8030-25df31ed8217 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:31.054157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4b215-6e16-4dca-b385-989388b54de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:31.054998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4b215-6e16-4dca-b385-989388b54de8 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:31.055753 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4b215-6e16-4dca-b385-989388b54de8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:31.056279 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4b215-6e16-4dca-b385-989388b54de8 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:31.057452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4b215-6e16-4dca-b385-989388b54de8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:31.058906 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4b215-6e16-4dca-b385-989388b54de8 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:31.190990 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 36/61] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:07:30 2022] POST /identity/v3/users => generated 275 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:07:35.165775 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1ac27-cc0c-4c4f-866f-9ab153fd5b27 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:35.166819 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1ac27-cc0c-4c4f-866f-9ab153fd5b27 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:35.167340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1ac27-cc0c-4c4f-866f-9ab153fd5b27 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:35.168967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 26/62] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:35.174756 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52886d6d-4754-4780-aa53-25abbf7295f7 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:35.175788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52886d6d-4754-4780-aa53-25abbf7295f7 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:35.176809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52886d6d-4754-4780-aa53-25abbf7295f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:35.210782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-52886d6d-4754-4780-aa53-25abbf7295f7 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:35.213122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-52886d6d-4754-4780-aa53-25abbf7295f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:35.334355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 37/63] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:35.456532 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab96b08-68bf-4f5c-94a9-64192df38496 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:35.457165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab96b08-68bf-4f5c-94a9-64192df38496 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:35.458307 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab96b08-68bf-4f5c-94a9-64192df38496 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:35.461281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 27/64] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:35.467236 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8bd0c-3d1e-4b10-96a3-2cba8b2dd58a None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:35.467859 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8bd0c-3d1e-4b10-96a3-2cba8b2dd58a None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:35.468349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8bd0c-3d1e-4b10-96a3-2cba8b2dd58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:35.496519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-48d8bd0c-3d1e-4b10-96a3-2cba8b2dd58a None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:35.498166 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-48d8bd0c-3d1e-4b10-96a3-2cba8b2dd58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:35.613687 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 38/65] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:35.685506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab96b08-68bf-4f5c-94a9-64192df38496 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:35.758964 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:35.945085 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab96b08-68bf-4f5c-94a9-64192df38496 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:35.948852 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8bd0c-3d1e-4b10-96a3-2cba8b2dd58a None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:35.966486 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:36.003958 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a35a65-8722-4cea-ba15-88ba8b47c996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:36.005726 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a35a65-8722-4cea-ba15-88ba8b47c996 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:36.006009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a35a65-8722-4cea-ba15-88ba8b47c996 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:36.006337 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a35a65-8722-4cea-ba15-88ba8b47c996 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:36.019659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a35a65-8722-4cea-ba15-88ba8b47c996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:36.025330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-41a35a65-8722-4cea-ba15-88ba8b47c996 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:07:36.025330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:36.025330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:36.025330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:36.025330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:36.025330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:36.025330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:36.025330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:36.025330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:36.025330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:36.025330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:36.025330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:36.025330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:36.025330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:36.025330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:36.025330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:36.026253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:36.026253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:36.026253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:36.026253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:36.026253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:36.026253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:36.026253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:36.026253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:36.026253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:36.026253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:36.026253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:36.026253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:36.026253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:36.026253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:36.026253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:36.026253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:36.027115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:36.027115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:36.027115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:36.027115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:36.027115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:36.027115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:36.027115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:36.027115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:36.027115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:36.027115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:36.027115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:07:36.027115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:07:36.027115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 28/66] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:07:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:07:36.027823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbbd43-e221-4da6-863f-e32e0ad8d3c4 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:36.028416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbbd43-e221-4da6-863f-e32e0ad8d3c4 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:36.028416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbbd43-e221-4da6-863f-e32e0ad8d3c4 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:36.029851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 29/67] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:36.033958 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c6166-7015-4a0b-ac36-198af2884c46 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:36.034245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c6166-7015-4a0b-ac36-198af2884c46 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:36.034529 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c6166-7015-4a0b-ac36-198af2884c46 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:36.037259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 30/68] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:36.038052 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c6166-7015-4a0b-ac36-198af2884c46 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:36.098884 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8bd0c-3d1e-4b10-96a3-2cba8b2dd58a None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:36.144031 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274dde6-b84f-49ba-b6e6-6b201cdf6f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:36.144703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274dde6-b84f-49ba-b6e6-6b201cdf6f02 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:36.144984 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274dde6-b84f-49ba-b6e6-6b201cdf6f02 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:36.145768 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274dde6-b84f-49ba-b6e6-6b201cdf6f02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:36.159282 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274dde6-b84f-49ba-b6e6-6b201cdf6f02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:36.165181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-6274dde6-b84f-49ba-b6e6-6b201cdf6f02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:07:36.165181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:36.165181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:36.165181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:36.165181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:36.165181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:36.165181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:36.165181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:36.165181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:36.165181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:36.165181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:36.165181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:36.165181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:36.165181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:36.165181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:36.165181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:36.166077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:36.166077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:36.166077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:36.166077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:36.166077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:36.166077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:36.166077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:36.166077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:36.166077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:36.166077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:36.166077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:36.166077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:36.166077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:36.166077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:36.166077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:36.166077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:36.167249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:36.167249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:36.167249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:36.167249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:36.167249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:36.167249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:36.167249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:36.167249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:36.167249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:36.167249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:36.167249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:07:36.167249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:07:36.167249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 39/69] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:07:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:07:36.168393 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624612e-1ef7-4444-b113-1f1e73f47a59 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:36.168736 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624612e-1ef7-4444-b113-1f1e73f47a59 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:36.169084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624612e-1ef7-4444-b113-1f1e73f47a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:36.178383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:36.190004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-3624612e-1ef7-4444-b113-1f1e73f47a59 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:36.195305 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-3624612e-1ef7-4444-b113-1f1e73f47a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:36.195442 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c6166-7015-4a0b-ac36-198af2884c46 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:36.247592 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a7613-a8c8-47c7-ad1d-2b31ae2b34f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:36.248240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a7613-a8c8-47c7-ad1d-2b31ae2b34f5 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:36.248524 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a7613-a8c8-47c7-ad1d-2b31ae2b34f5 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:36.248800 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a7613-a8c8-47c7-ad1d-2b31ae2b34f5 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:36.249502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a7613-a8c8-47c7-ad1d-2b31ae2b34f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:36.250472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a7613-a8c8-47c7-ad1d-2b31ae2b34f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:36.251310 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a7613-a8c8-47c7-ad1d-2b31ae2b34f5 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:36.261684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 31/70] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:07:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:36.262959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62174f65-5cae-4bd5-bc5c-18edffc9ee0c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:36.263276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62174f65-5cae-4bd5-bc5c-18edffc9ee0c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:36.263710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62174f65-5cae-4bd5-bc5c-18edffc9ee0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:36.280105 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:36.284591 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-62174f65-5cae-4bd5-bc5c-18edffc9ee0c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:36.286555 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-62174f65-5cae-4bd5-bc5c-18edffc9ee0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:36.287304 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 40/71] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:36.288251 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624612e-1ef7-4444-b113-1f1e73f47a59 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:36.365404 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 32/72] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:36.366698 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967bf1a-348c-4d60-89ad-5fa57124f38c None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:36.368027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967bf1a-348c-4d60-89ad-5fa57124f38c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:36.368027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967bf1a-348c-4d60-89ad-5fa57124f38c None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:36.368694 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 33/73] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:36.369595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967bf1a-348c-4d60-89ad-5fa57124f38c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:36.456566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624612e-1ef7-4444-b113-1f1e73f47a59 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:36.506669 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:36.509487 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d39cf5-0b0e-4ec2-8ce1-3af8f7fe69c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:36.510288 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d39cf5-0b0e-4ec2-8ce1-3af8f7fe69c1 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:36.510756 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d39cf5-0b0e-4ec2-8ce1-3af8f7fe69c1 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:36.511204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d39cf5-0b0e-4ec2-8ce1-3af8f7fe69c1 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:36.512032 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d39cf5-0b0e-4ec2-8ce1-3af8f7fe69c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:36.512484 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d39cf5-0b0e-4ec2-8ce1-3af8f7fe69c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:36.514148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d39cf5-0b0e-4ec2-8ce1-3af8f7fe69c1 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:36.530282 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 41/74] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:07:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:36.531972 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1df31-4f11-49e5-adcd-d0f418a401ee None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:36.532409 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1df31-4f11-49e5-adcd-d0f418a401ee None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:36.532825 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1df31-4f11-49e5-adcd-d0f418a401ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:36.534082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967bf1a-348c-4d60-89ad-5fa57124f38c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:36.558680 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-57b1df31-4f11-49e5-adcd-d0f418a401ee None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:36.560640 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-57b1df31-4f11-49e5-adcd-d0f418a401ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:36.598907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f29bb-f934-4775-8303-6e1eebc31da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:36.600331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f29bb-f934-4775-8303-6e1eebc31da3 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:36.601025 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f29bb-f934-4775-8303-6e1eebc31da3 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:36.601560 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f29bb-f934-4775-8303-6e1eebc31da3 None admin] PATH_INFO: `/v3/users/58c09285a32c4e3886cdd7b86f345a0c` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:36.618017 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f29bb-f934-4775-8303-6e1eebc31da3 None admin] RBAC: Authorizing `identity:get_user(user_id=58c09285a32c4e3886cdd7b86f345a0c)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:36.619858 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f29bb-f934-4775-8303-6e1eebc31da3 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:36.626203 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 34/75] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:36 2022] GET /identity/v3/users/58c09285a32c4e3886cdd7b86f345a0c => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:36.626203 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f29bb-f934-4775-8303-6e1eebc31da3 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:36.638953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:36.655015 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 42/76] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:36.656185 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1df31-4f11-49e5-adcd-d0f418a401ee None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:36.778697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f29bb-f934-4775-8303-6e1eebc31da3 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:36.805675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1df31-4f11-49e5-adcd-d0f418a401ee None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:36.825001 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0ef54-f448-4862-8926-3e522fdcbac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:36.825655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0ef54-f448-4862-8926-3e522fdcbac5 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:36.825937 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0ef54-f448-4862-8926-3e522fdcbac5 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:36.826332 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0ef54-f448-4862-8926-3e522fdcbac5 None admin] PATH_INFO: `/v3/users/7a8e484a77ce40f880d21a33a930f0be` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:36.827750 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0ef54-f448-4862-8926-3e522fdcbac5 None admin] RBAC: Authorizing `identity:get_user(user_id=7a8e484a77ce40f880d21a33a930f0be)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:36.829276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0ef54-f448-4862-8926-3e522fdcbac5 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:36.836690 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 35/77] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:36 2022] GET /identity/v3/users/7a8e484a77ce40f880d21a33a930f0be => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:36.836690 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0ef54-f448-4862-8926-3e522fdcbac5 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:36.847349 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:36.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f155cb-5a80-4621-ad48-6026750ab48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:36.864913 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f155cb-5a80-4621-ad48-6026750ab48a None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:36.865440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f155cb-5a80-4621-ad48-6026750ab48a None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:36.865949 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f155cb-5a80-4621-ad48-6026750ab48a None admin] PATH_INFO: `/v3/domains/default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:36.868373 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f155cb-5a80-4621-ad48-6026750ab48a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:36.870726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f155cb-5a80-4621-ad48-6026750ab48a None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:36.872746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 43/78] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:07:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:36.873337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f155cb-5a80-4621-ad48-6026750ab48a None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:36.939815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:36.964535 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0ef54-f448-4862-8926-3e522fdcbac5 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:36.986656 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f155cb-5a80-4621-ad48-6026750ab48a None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:37.010243 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8c8f2-6776-49cc-a43c-55026e6d95ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:37.011051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8c8f2-6776-49cc-a43c-55026e6d95ae None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:37.011376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8c8f2-6776-49cc-a43c-55026e6d95ae None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:37.011797 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8c8f2-6776-49cc-a43c-55026e6d95ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:37.040408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f951b6-6129-4844-9570-780dbbd01ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:37.041265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f951b6-6129-4844-9570-780dbbd01ebd None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:37.041543 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f951b6-6129-4844-9570-780dbbd01ebd None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:37.041851 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f951b6-6129-4844-9570-780dbbd01ebd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:37.053557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f951b6-6129-4844-9570-780dbbd01ebd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:37.058608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-f6f951b6-6129-4844-9570-780dbbd01ebd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:07:37.058608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:37.058608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:37.058608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:37.058608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:37.058608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:37.058608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:37.058608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:37.058608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:37.058608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:37.058608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:37.058608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:37.058608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:37.058608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:37.058608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:37.058608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:37.059873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:37.059873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:37.059873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:37.059873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:37.059873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:37.059873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:37.059873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:37.059873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:37.059873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:37.059873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:37.059873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:37.059873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:37.059873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:37.059873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:37.059873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:37.059873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:37.060696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:37.060696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:37.060696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:37.060696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:37.060696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:37.060696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:37.060696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:37.060696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:37.060696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:37.060696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:37.060696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:07:37.060696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:07:37.060696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 44/79] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:07:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:07:37.060696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f951b6-6129-4844-9570-780dbbd01ebd None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:37.095400 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8c8f2-6776-49cc-a43c-55026e6d95ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:37.096902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8c8f2-6776-49cc-a43c-55026e6d95ae None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:37.108137 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:37.179571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f951b6-6129-4844-9570-780dbbd01ebd None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:37.199019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:37.217419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 36/80] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:07:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:07:37.218287 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8c8f2-6776-49cc-a43c-55026e6d95ae None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:37.224474 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d584d0a-434a-446d-8824-681753a99537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:37.224747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d584d0a-434a-446d-8824-681753a99537 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:37.225016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d584d0a-434a-446d-8824-681753a99537 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:37.225336 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d584d0a-434a-446d-8824-681753a99537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:37.320663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d584d0a-434a-446d-8824-681753a99537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:37.322284 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d584d0a-434a-446d-8824-681753a99537 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:37.368860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8c8f2-6776-49cc-a43c-55026e6d95ae None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:37.379285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:37.406269 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65656c8-6531-4fc4-bf9d-3facc5e43b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:37.406895 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65656c8-6531-4fc4-bf9d-3facc5e43b0e None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:37.407210 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65656c8-6531-4fc4-bf9d-3facc5e43b0e None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:37.407494 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65656c8-6531-4fc4-bf9d-3facc5e43b0e None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:37.408163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65656c8-6531-4fc4-bf9d-3facc5e43b0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:37.409684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65656c8-6531-4fc4-bf9d-3facc5e43b0e None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:37.438324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 45/81] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:07:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:07:37.441123 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d584d0a-434a-446d-8824-681753a99537 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:37.461765 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:37.496754 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 37/82] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:07:37 2022] POST /identity/v3/users => generated 289 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:37.497636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65656c8-6531-4fc4-bf9d-3facc5e43b0e None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:37.596324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d584d0a-434a-446d-8824-681753a99537 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:37.659172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d83bcb-6d98-4b3d-abff-f0d6fcf06894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:37.660058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d83bcb-6d98-4b3d-abff-f0d6fcf06894 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:37.660570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d83bcb-6d98-4b3d-abff-f0d6fcf06894 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:37.661582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d83bcb-6d98-4b3d-abff-f0d6fcf06894 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:37.665975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65656c8-6531-4fc4-bf9d-3facc5e43b0e None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:37.676207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d83bcb-6d98-4b3d-abff-f0d6fcf06894 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:37.681813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-41d83bcb-6d98-4b3d-abff-f0d6fcf06894 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:07:37.681813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:37.681813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:37.681813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:37.681813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:37.681813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:37.681813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:37.681813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:37.681813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:37.681813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:37.681813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:37.681813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:37.681813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:37.681813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:37.681813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:37.681813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:37.687835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:37.687835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:37.687835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:37.687835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:37.687835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:37.687835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:37.687835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:37.687835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:37.687835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:37.687835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:37.687835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:37.687835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:37.687835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:37.687835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:37.687835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:37.687835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:37.688626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:37.688626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:37.688626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:37.688626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:37.688626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:37.688626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:37.688626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:37.688626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:37.688626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:37.688626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:37.688626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:07:37.688626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:07:37.688626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 46/83] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:07:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:07:37.688626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d83bcb-6d98-4b3d-abff-f0d6fcf06894 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:37.732872 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a832da3-c940-4813-915a-4d63ad0018a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:37.733504 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a832da3-c940-4813-915a-4d63ad0018a6 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:37.733790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a832da3-c940-4813-915a-4d63ad0018a6 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:37.735289 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a832da3-c940-4813-915a-4d63ad0018a6 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:37.735289 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a832da3-c940-4813-915a-4d63ad0018a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:37.735289 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a832da3-c940-4813-915a-4d63ad0018a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:37.736572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a832da3-c940-4813-915a-4d63ad0018a6 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:37.747109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 38/84] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:07:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:37.748752 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a832da3-c940-4813-915a-4d63ad0018a6 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:37.795806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:37.811273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:37.859082 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d83bcb-6d98-4b3d-abff-f0d6fcf06894 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:37.921024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a832da3-c940-4813-915a-4d63ad0018a6 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:37.925019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4074bb5-1bdf-44ed-9e43-fc8dd3093afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:37.925674 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4074bb5-1bdf-44ed-9e43-fc8dd3093afb None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:37.925951 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4074bb5-1bdf-44ed-9e43-fc8dd3093afb None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:37.926918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4074bb5-1bdf-44ed-9e43-fc8dd3093afb None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:37.928179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4074bb5-1bdf-44ed-9e43-fc8dd3093afb None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:37.930544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4074bb5-1bdf-44ed-9e43-fc8dd3093afb None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:37.931688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 47/85] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:07:37 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:37.932176 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4074bb5-1bdf-44ed-9e43-fc8dd3093afb None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:37.989935 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f9fff-5c30-42b4-9d8f-f19ab5906c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:37.990819 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f9fff-5c30-42b4-9d8f-f19ab5906c59 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:37.991298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f9fff-5c30-42b4-9d8f-f19ab5906c59 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:37.991769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f9fff-5c30-42b4-9d8f-f19ab5906c59 None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:37.994447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f9fff-5c30-42b4-9d8f-f19ab5906c59 None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:37.997260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f9fff-5c30-42b4-9d8f-f19ab5906c59 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:38.000687 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 39/86] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:07:37 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:38.001918 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f9fff-5c30-42b4-9d8f-f19ab5906c59 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:38.094053 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4074bb5-1bdf-44ed-9e43-fc8dd3093afb None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:38.123367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:38.131469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f9fff-5c30-42b4-9d8f-f19ab5906c59 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:38.145514 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173aa2a-e3c4-4975-9a82-c463161bc457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:38.146271 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173aa2a-e3c4-4975-9a82-c463161bc457 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:38.146625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173aa2a-e3c4-4975-9a82-c463161bc457 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:38.146910 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173aa2a-e3c4-4975-9a82-c463161bc457 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:38.147654 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173aa2a-e3c4-4975-9a82-c463161bc457 None admin] RBAC: Adding query filter params (name=member) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:38.147966 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173aa2a-e3c4-4975-9a82-c463161bc457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:38.149380 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173aa2a-e3c4-4975-9a82-c463161bc457 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:38.159049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 48/87] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:07:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:38.160122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173aa2a-e3c4-4975-9a82-c463161bc457 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:38.173797 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:38.177252 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56c6d1-b215-4d82-b9cc-23dcc4bab3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:38.178044 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56c6d1-b215-4d82-b9cc-23dcc4bab3b2 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:38.178268 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56c6d1-b215-4d82-b9cc-23dcc4bab3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:38.178594 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56c6d1-b215-4d82-b9cc-23dcc4bab3b2 None admin] PATH_INFO: `/v3/users/58c09285a32c4e3886cdd7b86f345a0c` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:38.179931 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56c6d1-b215-4d82-b9cc-23dcc4bab3b2 None admin] RBAC: Authorizing `identity:get_user(user_id=58c09285a32c4e3886cdd7b86f345a0c)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:38.182053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56c6d1-b215-4d82-b9cc-23dcc4bab3b2 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:38.183791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 40/88] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:38 2022] GET /identity/v3/users/58c09285a32c4e3886cdd7b86f345a0c => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:38.187846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56c6d1-b215-4d82-b9cc-23dcc4bab3b2 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:38.296771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173aa2a-e3c4-4975-9a82-c463161bc457 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:38.356712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230f9e4-52f9-42cc-ad59-517db7891dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:38.357410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230f9e4-52f9-42cc-ad59-517db7891dea None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:38.357734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230f9e4-52f9-42cc-ad59-517db7891dea None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:38.358019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230f9e4-52f9-42cc-ad59-517db7891dea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:38.358843 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230f9e4-52f9-42cc-ad59-517db7891dea None admin] RBAC: Adding query filter params (role.id=bcc52f73cf03442e922a4757908334de, scope.project.id=1fcdd13492724455ae65d6e80d2f41bc, user.id=58c09285a32c4e3886cdd7b86f345a0c) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:38.359594 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230f9e4-52f9-42cc-ad59-517db7891dea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:38.361824 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230f9e4-52f9-42cc-ad59-517db7891dea None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:38.379297 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 49/89] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:07:38 2022] GET /identity/v3/role_assignments?user.id=58c09285a32c4e3886cdd7b86f345a0c&scope.project.id=1fcdd13492724455ae65d6e80d2f41bc&role.id=bcc52f73cf03442e922a4757908334de => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:38.379297 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230f9e4-52f9-42cc-ad59-517db7891dea None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:38.425046 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56c6d1-b215-4d82-b9cc-23dcc4bab3b2 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:38.495514 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:38.552082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c4732-720c-4f4d-9ecf-806dca2d6e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:38.553074 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c4732-720c-4f4d-9ecf-806dca2d6e61 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:38.553549 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c4732-720c-4f4d-9ecf-806dca2d6e61 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:38.553873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c4732-720c-4f4d-9ecf-806dca2d6e61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:38.554854 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c4732-720c-4f4d-9ecf-806dca2d6e61 None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=1fcdd13492724455ae65d6e80d2f41bc, user.id=7a8e484a77ce40f880d21a33a930f0be) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:38.555439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c4732-720c-4f4d-9ecf-806dca2d6e61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:38.557043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c4732-720c-4f4d-9ecf-806dca2d6e61 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:38.569303 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230f9e4-52f9-42cc-ad59-517db7891dea None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:38.579667 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 41/90] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:07:38 2022] GET /identity/v3/role_assignments?user.id=7a8e484a77ce40f880d21a33a930f0be&scope.project.id=1fcdd13492724455ae65d6e80d2f41bc&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 259 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:38.581197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c4732-720c-4f4d-9ecf-806dca2d6e61 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:38.665174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773927ff-2872-4219-9150-63b90bd8da6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:38.665702 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:38.667701 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773927ff-2872-4219-9150-63b90bd8da6a None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:38.668581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773927ff-2872-4219-9150-63b90bd8da6a None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:38.670759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773927ff-2872-4219-9150-63b90bd8da6a None admin] PATH_INFO: `/v3/users/58c09285a32c4e3886cdd7b86f345a0c` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:38.674081 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773927ff-2872-4219-9150-63b90bd8da6a None admin] RBAC: Authorizing `identity:get_user(user_id=58c09285a32c4e3886cdd7b86f345a0c)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:38.679230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773927ff-2872-4219-9150-63b90bd8da6a None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:38.682652 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 50/91] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:38 2022] GET /identity/v3/users/58c09285a32c4e3886cdd7b86f345a0c => generated 275 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:38.693314 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773927ff-2872-4219-9150-63b90bd8da6a None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:38.886814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c4732-720c-4f4d-9ecf-806dca2d6e61 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:38.948228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773927ff-2872-4219-9150-63b90bd8da6a None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:38.984251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d89629-2085-4c30-bb14-106930276589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:38.985528 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d89629-2085-4c30-bb14-106930276589 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:38.986046 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d89629-2085-4c30-bb14-106930276589 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:38.986541 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d89629-2085-4c30-bb14-106930276589 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:38.992580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb1c4c-ecd0-40da-8670-17839f9938e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:38.993214 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb1c4c-ecd0-40da-8670-17839f9938e5 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:38.993496 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb1c4c-ecd0-40da-8670-17839f9938e5 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:38.993768 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb1c4c-ecd0-40da-8670-17839f9938e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:39.013408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:39.127150 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb1c4c-ecd0-40da-8670-17839f9938e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:39.128753 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb1c4c-ecd0-40da-8670-17839f9938e5 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:39.155640 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d89629-2085-4c30-bb14-106930276589 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:39.157205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d89629-2085-4c30-bb14-106930276589 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:39.239824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:39.375628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 51/92] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:07:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:07:39.387163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb1c4c-ecd0-40da-8670-17839f9938e5 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:39.423368 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 42/93] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:07:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 843 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:07:39.432210 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d89629-2085-4c30-bb14-106930276589 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:39.539812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:39.709429 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d89629-2085-4c30-bb14-106930276589 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:39.739867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb1c4c-ecd0-40da-8670-17839f9938e5 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:39.767014 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:39.795739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c907e1-824e-4e6c-af70-9d1c4e48a379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:39.797326 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c907e1-824e-4e6c-af70-9d1c4e48a379 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:39.798118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c907e1-824e-4e6c-af70-9d1c4e48a379 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:39.798882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c907e1-824e-4e6c-af70-9d1c4e48a379 None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:39.804750 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c907e1-824e-4e6c-af70-9d1c4e48a379 None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:39.808099 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c907e1-824e-4e6c-af70-9d1c4e48a379 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:39.812211 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 43/94] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:07:39 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:39.821919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c907e1-824e-4e6c-af70-9d1c4e48a379 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:39.828349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72e62f-e23c-4e32-85f9-32e8de98ce84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:39.829018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72e62f-e23c-4e32-85f9-32e8de98ce84 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:39.829299 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72e62f-e23c-4e32-85f9-32e8de98ce84 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:39.829571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72e62f-e23c-4e32-85f9-32e8de98ce84 None admin] PATH_INFO: `/v3/projects/a4400166f2f64a2a9b6a52c84af30069` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:39.836562 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e72e62f-e23c-4e32-85f9-32e8de98ce84 None admin] RBAC: Authorizing `identity:get_project(project_id=a4400166f2f64a2a9b6a52c84af30069)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:39.837580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e72e62f-e23c-4e32-85f9-32e8de98ce84 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:39.839764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 52/95] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:07:39 2022] GET /identity/v3/projects/a4400166f2f64a2a9b6a52c84af30069 => generated 312 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:39.852052 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72e62f-e23c-4e32-85f9-32e8de98ce84 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:40.089324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72e62f-e23c-4e32-85f9-32e8de98ce84 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:40.110357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:40.155688 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c907e1-824e-4e6c-af70-9d1c4e48a379 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:40.194417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b58d9-308f-45f8-9744-7cc1ed92627b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:40.196046 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b58d9-308f-45f8-9744-7cc1ed92627b None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:40.196804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b58d9-308f-45f8-9744-7cc1ed92627b None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:40.197508 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b58d9-308f-45f8-9744-7cc1ed92627b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:40.199248 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b58d9-308f-45f8-9744-7cc1ed92627b None admin] RBAC: Adding query filter params (role.id=bcc52f73cf03442e922a4757908334de, scope.project.id=a4400166f2f64a2a9b6a52c84af30069, user.id=58c09285a32c4e3886cdd7b86f345a0c) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:40.200084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b58d9-308f-45f8-9744-7cc1ed92627b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:40.204064 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b58d9-308f-45f8-9744-7cc1ed92627b None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:40.223499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 53/96] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:07:39 2022] GET /identity/v3/role_assignments?user.id=58c09285a32c4e3886cdd7b86f345a0c&scope.project.id=a4400166f2f64a2a9b6a52c84af30069&role.id=bcc52f73cf03442e922a4757908334de => generated 259 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:40.253459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1a2df-2f06-4bf5-9425-c60123c9c9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:40.254103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1a2df-2f06-4bf5-9425-c60123c9c9ef None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:40.254901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1a2df-2f06-4bf5-9425-c60123c9c9ef None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:40.255384 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1a2df-2f06-4bf5-9425-c60123c9c9ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:40.256525 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1a2df-2f06-4bf5-9425-c60123c9c9ef None admin] RBAC: Adding query filter params (role.id=fc0a7877dd0f4903a2a6fcc01a829c6a, scope.project.id=1fcdd13492724455ae65d6e80d2f41bc, user.id=58c09285a32c4e3886cdd7b86f345a0c) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:40.257011 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1a2df-2f06-4bf5-9425-c60123c9c9ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:40.258652 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1a2df-2f06-4bf5-9425-c60123c9c9ef None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:40.274685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 44/97] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:07:39 2022] GET /identity/v3/role_assignments?user.id=58c09285a32c4e3886cdd7b86f345a0c&scope.project.id=1fcdd13492724455ae65d6e80d2f41bc&role.id=fc0a7877dd0f4903a2a6fcc01a829c6a => generated 259 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:41.371083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1753f8-ced2-41ac-9e79-3d43a61d74cc None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:41.371861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1753f8-ced2-41ac-9e79-3d43a61d74cc None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:41.371861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1753f8-ced2-41ac-9e79-3d43a61d74cc None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:41.373050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 54/98] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:41.380582 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf96fd5-05eb-45df-a7fb-b2c03804501e None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:41.382144 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf96fd5-05eb-45df-a7fb-b2c03804501e None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:41.382919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf96fd5-05eb-45df-a7fb-b2c03804501e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:41.423841 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-caf96fd5-05eb-45df-a7fb-b2c03804501e None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:41.427215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-caf96fd5-05eb-45df-a7fb-b2c03804501e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:41.582707 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:41.624139 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 45/99] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:42.159896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1753f8-ced2-41ac-9e79-3d43a61d74cc None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:42.295755 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38f06d-9d04-424f-aede-52cd8a5eeca6 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:42.296026 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38f06d-9d04-424f-aede-52cd8a5eeca6 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:42.296418 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38f06d-9d04-424f-aede-52cd8a5eeca6 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:42.303196 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 46/100] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:42.310455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe3825-909e-42b7-8b0d-40857ed48b59 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:42.310726 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe3825-909e-42b7-8b0d-40857ed48b59 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:42.311034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe3825-909e-42b7-8b0d-40857ed48b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:42.333394 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1753f8-ced2-41ac-9e79-3d43a61d74cc None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:42.365448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-2dbe3825-909e-42b7-8b0d-40857ed48b59 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:42.368133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2dbe3825-909e-42b7-8b0d-40857ed48b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:42.409828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd89a9c-fc42-43a3-b3f5-d4af36821c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:42.411040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd89a9c-fc42-43a3-b3f5-d4af36821c94 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:42.411465 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd89a9c-fc42-43a3-b3f5-d4af36821c94 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:42.411919 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd89a9c-fc42-43a3-b3f5-d4af36821c94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:42.416108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd89a9c-fc42-43a3-b3f5-d4af36821c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:42.418401 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd89a9c-fc42-43a3-b3f5-d4af36821c94 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:42.421100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 55/101] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:07:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:42.437537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd89a9c-fc42-43a3-b3f5-d4af36821c94 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:42.511854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:42.587801 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd89a9c-fc42-43a3-b3f5-d4af36821c94 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:42.596492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 47/102] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 07 13:07:42.634003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9e8fd-e32d-4360-8156-183f42fc3e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:42.635048 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9e8fd-e32d-4360-8156-183f42fc3e52 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:42.635536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9e8fd-e32d-4360-8156-183f42fc3e52 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:42.636024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9e8fd-e32d-4360-8156-183f42fc3e52 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:42.637180 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9e8fd-e32d-4360-8156-183f42fc3e52 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:42.638991 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9e8fd-e32d-4360-8156-183f42fc3e52 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:42.731747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cd612-c442-4ab9-8160-a6d86c0048a2 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:42.731980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cd612-c442-4ab9-8160-a6d86c0048a2 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:42.732465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cd612-c442-4ab9-8160-a6d86c0048a2 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:42.734148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 48/103] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:07:42.740707 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cb82b-b405-459b-befc-1901429aca4d None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:42.741368 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cb82b-b405-459b-befc-1901429aca4d None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:42.741850 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cb82b-b405-459b-befc-1901429aca4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:42.752598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 56/104] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 7 13:07:42 2022] POST /identity/v3/projects => generated 302 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:07:42.777555 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-949cb82b-b405-459b-befc-1901429aca4d None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:42.780450 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-949cb82b-b405-459b-befc-1901429aca4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:42.934626 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 49/105] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:07:42.946328 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fa335-2712-46d3-8462-0a206d2f824f None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:42.947146 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fa335-2712-46d3-8462-0a206d2f824f None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:42.947681 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fa335-2712-46d3-8462-0a206d2f824f None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:42.949509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 57/106] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:42.958214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4dcfd1-f87c-4247-9fe0-c14088ec812d None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:42.959458 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4dcfd1-f87c-4247-9fe0-c14088ec812d None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:42.960242 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4dcfd1-f87c-4247-9fe0-c14088ec812d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:42.988317 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-4b4dcfd1-f87c-4247-9fe0-c14088ec812d None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:42.990856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-4b4dcfd1-f87c-4247-9fe0-c14088ec812d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:43.068939 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:43.085370 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fa335-2712-46d3-8462-0a206d2f824f None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:43.147240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 50/107] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:43.272338 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:43.287156 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fa335-2712-46d3-8462-0a206d2f824f None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:43.325816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4dcfd1-f87c-4247-9fe0-c14088ec812d None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:43.355084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9407cf9-8c7a-4694-9439-7923b509df9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:43.356592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9407cf9-8c7a-4694-9439-7923b509df9e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:43.357082 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9407cf9-8c7a-4694-9439-7923b509df9e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:43.357565 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9407cf9-8c7a-4694-9439-7923b509df9e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:43.391816 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9407cf9-8c7a-4694-9439-7923b509df9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:43.403892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-b9407cf9-8c7a-4694-9439-7923b509df9e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:07:43.403892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:43.403892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:43.403892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:43.403892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:43.403892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:43.403892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:43.403892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:43.403892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:43.403892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:43.403892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:43.403892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:43.403892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:43.403892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:43.403892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:43.403892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:43.405022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:43.405022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:43.405022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:43.405022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:43.405022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:43.405022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:43.405022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:43.405022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:43.405022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:43.405022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:43.405022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:43.405022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:43.405022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:43.405022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:43.405022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:43.405022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:43.405792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:43.405792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:43.405792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:43.405792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:43.405792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:43.405792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:43.405792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:43.405792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:43.405792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:43.405792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:43.405792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:07:43.405792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:07:43.405792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 58/108] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:07:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:07:43.412414 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9407cf9-8c7a-4694-9439-7923b509df9e None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:43.559710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4dcfd1-f87c-4247-9fe0-c14088ec812d None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:43.652170 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9407cf9-8c7a-4694-9439-7923b509df9e None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:43.687889 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f9b6e-eff4-4030-afb8-b374fef67150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:43.688708 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f9b6e-eff4-4030-afb8-b374fef67150 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:43.688911 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f9b6e-eff4-4030-afb8-b374fef67150 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:43.689440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f9b6e-eff4-4030-afb8-b374fef67150 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:43.719962 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129756d2-56ca-4cfa-9bfe-cda1333a96f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:43.720867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129756d2-56ca-4cfa-9bfe-cda1333a96f5 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:43.721277 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129756d2-56ca-4cfa-9bfe-cda1333a96f5 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:43.721724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129756d2-56ca-4cfa-9bfe-cda1333a96f5 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:43.722821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129756d2-56ca-4cfa-9bfe-cda1333a96f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:43.724049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129756d2-56ca-4cfa-9bfe-cda1333a96f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:43.726222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129756d2-56ca-4cfa-9bfe-cda1333a96f5 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:43.734552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f9b6e-eff4-4030-afb8-b374fef67150 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:43.742731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-352f9b6e-eff4-4030-afb8-b374fef67150 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:07:43.742731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:43.742731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:43.742731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:43.742731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:43.742731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:43.742731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:43.742731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:43.742731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:43.742731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:43.742731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:43.742731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:43.742731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:43.742731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:43.742731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:43.742731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:43.743747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:43.743747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:43.743747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:43.743747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:43.743747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:43.743747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:43.743747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:43.743747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:43.743747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:43.743747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:43.743747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:43.743747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:43.743747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:43.743747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:43.743747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:43.743747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:43.744553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:43.744553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:43.744553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:43.744553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:43.744553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:43.744553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:43.744553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:43.744553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:43.744553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:43.744553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:43.744553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:07:43.744553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:07:43.751403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 51/109] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:07:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:07:43.752971 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f9b6e-eff4-4030-afb8-b374fef67150 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:43.760409 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 59/110] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:07:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:43.763749 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129756d2-56ca-4cfa-9bfe-cda1333a96f5 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:43.814382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:43.879722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:43.997492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f9b6e-eff4-4030-afb8-b374fef67150 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:44.018268 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129756d2-56ca-4cfa-9bfe-cda1333a96f5 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:44.086919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b17fd-7538-40d7-b153-230294a2afce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:44.088741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b17fd-7538-40d7-b153-230294a2afce None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:44.089028 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b17fd-7538-40d7-b153-230294a2afce None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:44.089307 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b17fd-7538-40d7-b153-230294a2afce None admin] PATH_INFO: `/v3/roles/member` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:44.104815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa79bf2-acee-4c1b-bf97-f81cb600a9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:44.105455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa79bf2-acee-4c1b-bf97-f81cb600a9cf None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:44.105920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa79bf2-acee-4c1b-bf97-f81cb600a9cf None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:44.106210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa79bf2-acee-4c1b-bf97-f81cb600a9cf None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:44.106929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa79bf2-acee-4c1b-bf97-f81cb600a9cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:44.107338 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa79bf2-acee-4c1b-bf97-f81cb600a9cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:44.108918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa79bf2-acee-4c1b-bf97-f81cb600a9cf None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:44.116417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b17fd-7538-40d7-b153-230294a2afce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:44.125548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-bd0b17fd-7538-40d7-b153-230294a2afce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:07:44.125548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:44.125548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:44.125548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:44.125548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:44.125548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:44.125548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:44.125548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:44.125548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:44.125548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:44.125548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:44.125548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:44.125548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:44.125548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:44.125548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:44.125548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:44.127747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:44.127747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:44.127747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:44.127747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:44.127747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:44.127747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:44.127747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:44.127747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:44.127747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:44.127747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:44.127747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:44.127747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:44.127747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:44.127747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:44.127747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:44.127747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:44.130445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:44.130445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:44.130445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:44.130445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:44.130445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:44.130445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:44.130445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:44.130445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:44.130445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:44.130445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:44.130445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:07:44.130445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:07:44.130445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 52/111] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:07:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:07:44.130445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b17fd-7538-40d7-b153-230294a2afce None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:44.141429 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 60/112] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:07:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:44.142951 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa79bf2-acee-4c1b-bf97-f81cb600a9cf None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:44.308895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:44.349597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa79bf2-acee-4c1b-bf97-f81cb600a9cf None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:44.406915 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90929b-50c7-46b3-9486-3fdf11363e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:44.408954 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90929b-50c7-46b3-9486-3fdf11363e02 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:44.409245 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90929b-50c7-46b3-9486-3fdf11363e02 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:44.409519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90929b-50c7-46b3-9486-3fdf11363e02 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:44.410212 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90929b-50c7-46b3-9486-3fdf11363e02 None admin] RBAC: Adding query filter params (name=member) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:44.411308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90929b-50c7-46b3-9486-3fdf11363e02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:44.412961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90929b-50c7-46b3-9486-3fdf11363e02 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:44.435538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:44.437740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 61/113] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:07:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:44.439122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90929b-50c7-46b3-9486-3fdf11363e02 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:44.440706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b17fd-7538-40d7-b153-230294a2afce None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:44.503620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639cb1b-00fd-4873-abce-abb4867322cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:44.505538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639cb1b-00fd-4873-abce-abb4867322cc None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:44.506084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639cb1b-00fd-4873-abce-abb4867322cc None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:44.506675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639cb1b-00fd-4873-abce-abb4867322cc None admin] PATH_INFO: `/v3/users/b04f44fc2e374957b021d2b4110e2056` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:44.537952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639cb1b-00fd-4873-abce-abb4867322cc None admin] RBAC: Authorizing `identity:get_user(user_id=b04f44fc2e374957b021d2b4110e2056)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:44.541670 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639cb1b-00fd-4873-abce-abb4867322cc None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:44.546051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 53/114] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:44 2022] GET /identity/v3/users/b04f44fc2e374957b021d2b4110e2056 => generated 289 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:44.549016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afedc7b-22e8-42e9-b584-7014ad3e43fd None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:44.549511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afedc7b-22e8-42e9-b584-7014ad3e43fd None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:44.550012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afedc7b-22e8-42e9-b584-7014ad3e43fd None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:44.553420 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 54/115] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:44.554841 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afedc7b-22e8-42e9-b584-7014ad3e43fd None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:44.667272 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90929b-50c7-46b3-9486-3fdf11363e02 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:44.707756 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afedc7b-22e8-42e9-b584-7014ad3e43fd None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:44.734155 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2e376-3edb-4b48-af3e-781964f5ed7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:44.735294 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2e376-3edb-4b48-af3e-781964f5ed7f None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:44.735799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2e376-3edb-4b48-af3e-781964f5ed7f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:44.736453 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2e376-3edb-4b48-af3e-781964f5ed7f None admin] PATH_INFO: `/v3/users/7a8e484a77ce40f880d21a33a930f0be` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:44.739398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2e376-3edb-4b48-af3e-781964f5ed7f None admin] RBAC: Authorizing `identity:get_user(user_id=7a8e484a77ce40f880d21a33a930f0be)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:44.742396 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2e376-3edb-4b48-af3e-781964f5ed7f None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:44.745969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 62/116] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:44 2022] GET /identity/v3/users/7a8e484a77ce40f880d21a33a930f0be => generated 247 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:44.747708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2e376-3edb-4b48-af3e-781964f5ed7f None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:44.756413 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80062c2-3f4d-4f4a-842a-aa9489bca432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:44.757063 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80062c2-3f4d-4f4a-842a-aa9489bca432 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:44.757455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80062c2-3f4d-4f4a-842a-aa9489bca432 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:44.757618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80062c2-3f4d-4f4a-842a-aa9489bca432 None admin] PATH_INFO: `/v3/users/58c09285a32c4e3886cdd7b86f345a0c` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:44.758838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80062c2-3f4d-4f4a-842a-aa9489bca432 None admin] RBAC: Authorizing `identity:get_user(user_id=58c09285a32c4e3886cdd7b86f345a0c)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:44.760294 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:44.760413 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80062c2-3f4d-4f4a-842a-aa9489bca432 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:44.762120 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 55/117] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:44 2022] GET /identity/v3/users/58c09285a32c4e3886cdd7b86f345a0c => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:44.763137 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdc107-d3d1-4b98-8422-2edf464de060 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:44.763410 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdc107-d3d1-4b98-8422-2edf464de060 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:44.763684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdc107-d3d1-4b98-8422-2edf464de060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:44.783671 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-2efdc107-d3d1-4b98-8422-2edf464de060 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:44.784891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2efdc107-d3d1-4b98-8422-2edf464de060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:44.806894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:44.872842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 56/118] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:44.874096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdc107-d3d1-4b98-8422-2edf464de060 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:44.899832 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2e376-3edb-4b48-af3e-781964f5ed7f None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:44.947106 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f06cd4-2574-48b1-88ae-aa878aa0ebc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:44.948572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f06cd4-2574-48b1-88ae-aa878aa0ebc6 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:44.949200 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f06cd4-2574-48b1-88ae-aa878aa0ebc6 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:44.949755 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f06cd4-2574-48b1-88ae-aa878aa0ebc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:45.027454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdc107-d3d1-4b98-8422-2edf464de060 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:45.037448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f06cd4-2574-48b1-88ae-aa878aa0ebc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:45.039362 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f06cd4-2574-48b1-88ae-aa878aa0ebc6 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:45.049558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:45.065508 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9e074-0576-49c4-a355-d26e77931eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:45.066159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9e074-0576-49c4-a355-d26e77931eca None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:45.066477 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9e074-0576-49c4-a355-d26e77931eca None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:45.067523 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9e074-0576-49c4-a355-d26e77931eca None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:45.068828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9e074-0576-49c4-a355-d26e77931eca None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:45.070423 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9e074-0576-49c4-a355-d26e77931eca None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:45.072228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 57/119] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:07:44 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:45.078252 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba1020-e3d9-4769-b30f-1bc6c05f2dee None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:45.078511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba1020-e3d9-4769-b30f-1bc6c05f2dee None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:45.078804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba1020-e3d9-4769-b30f-1bc6c05f2dee None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:45.080174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 58/120] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:45.083048 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba1020-e3d9-4769-b30f-1bc6c05f2dee None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:45.114458 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:45.182162 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 63/121] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:07:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:07:45.183210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f06cd4-2574-48b1-88ae-aa878aa0ebc6 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:45.217312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba1020-e3d9-4769-b30f-1bc6c05f2dee None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:45.273231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071e059-652a-4c1a-a15f-ccad71cfd8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:45.274281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071e059-652a-4c1a-a15f-ccad71cfd8f2 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:45.274719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071e059-652a-4c1a-a15f-ccad71cfd8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:45.275131 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071e059-652a-4c1a-a15f-ccad71cfd8f2 None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:45.276791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a071e059-652a-4c1a-a15f-ccad71cfd8f2 None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:45.279352 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a071e059-652a-4c1a-a15f-ccad71cfd8f2 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:45.282498 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 59/122] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:07:45 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:45.284356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05fca0-3787-4e92-9c2b-47b3c2c69d02 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:45.284863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05fca0-3787-4e92-9c2b-47b3c2c69d02 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:45.285449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05fca0-3787-4e92-9c2b-47b3c2c69d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:45.315317 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-7f05fca0-3787-4e92-9c2b-47b3c2c69d02 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:45.318011 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-7f05fca0-3787-4e92-9c2b-47b3c2c69d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:45.328130 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f06cd4-2574-48b1-88ae-aa878aa0ebc6 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:45.338425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:45.381143 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c03c2-2964-44d7-98bd-cbd42e918668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:45.381923 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c03c2-2964-44d7-98bd-cbd42e918668 None admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:45.382219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c03c2-2964-44d7-98bd-cbd42e918668 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:45.382504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c03c2-2964-44d7-98bd-cbd42e918668 None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc/users/7a8e484a77ce40f880d21a33a930f0be/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:45.384565 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c03c2-2964-44d7-98bd-cbd42e918668 None admin] RBAC: Authorizing `identity:create_grant(project_id=1fcdd13492724455ae65d6e80d2f41bc, user_id=7a8e484a77ce40f880d21a33a930f0be, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:45.386033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c03c2-2964-44d7-98bd-cbd42e918668 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:45.428860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 60/123] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:45.439245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05fca0-3787-4e92-9c2b-47b3c2c69d02 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:45.475139 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:45.481177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 64/124] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:07:45 2022] PUT /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc/users/7a8e484a77ce40f880d21a33a930f0be/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:45.482167 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c03c2-2964-44d7-98bd-cbd42e918668 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:45.647995 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c03c2-2964-44d7-98bd-cbd42e918668 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:45.738708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e589b2c-c6a3-4d1c-a364-1c5be71ec837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:45.739228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e589b2c-c6a3-4d1c-a364-1c5be71ec837 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:45.739545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e589b2c-c6a3-4d1c-a364-1c5be71ec837 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:45.740039 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e589b2c-c6a3-4d1c-a364-1c5be71ec837 None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:45.741767 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e589b2c-c6a3-4d1c-a364-1c5be71ec837 None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:45.743942 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e589b2c-c6a3-4d1c-a364-1c5be71ec837 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:45.747072 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 65/125] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:07:45 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:45.748820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e589b2c-c6a3-4d1c-a364-1c5be71ec837 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:45.766537 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05fca0-3787-4e92-9c2b-47b3c2c69d02 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:45.807673 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:45.818437 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152a651-1078-426e-b5dc-1724eabdf0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:45.819949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152a651-1078-426e-b5dc-1724eabdf0c9 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:45.825534 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152a651-1078-426e-b5dc-1724eabdf0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:45.825534 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152a651-1078-426e-b5dc-1724eabdf0c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:45.843090 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0152a651-1078-426e-b5dc-1724eabdf0c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:45.849403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-0152a651-1078-426e-b5dc-1724eabdf0c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:07:45.849403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:45.849403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:45.849403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:45.849403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:45.849403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:45.849403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:45.849403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:45.849403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:45.849403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:45.849403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:45.849403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:45.849403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:45.849403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:45.849403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:45.849403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:45.850330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:45.850330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:45.850330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:45.850330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:45.850330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:45.850330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:45.850330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:45.850330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:45.850330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:45.850330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:45.850330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:45.850330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:45.850330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:45.850330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:45.850330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:45.850330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:45.851122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:45.851122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:45.851122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:45.851122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:45.851122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:45.851122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:45.851122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:45.851122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:45.851122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:45.851122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:45.851122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:07:45.851122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:07:45.854058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 61/126] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:07:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:07:45.855352 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152a651-1078-426e-b5dc-1724eabdf0c9 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:45.921832 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e589b2c-c6a3-4d1c-a364-1c5be71ec837 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:45.994060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177c268-2b42-47c7-bdec-0a3c5ee2448d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:45.995064 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177c268-2b42-47c7-bdec-0a3c5ee2448d None admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:45.995488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177c268-2b42-47c7-bdec-0a3c5ee2448d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:45.995896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177c268-2b42-47c7-bdec-0a3c5ee2448d None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc/users/58c09285a32c4e3886cdd7b86f345a0c/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:45.998645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177c268-2b42-47c7-bdec-0a3c5ee2448d None admin] RBAC: Authorizing `identity:create_grant(project_id=1fcdd13492724455ae65d6e80d2f41bc, user_id=58c09285a32c4e3886cdd7b86f345a0c, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:46.001680 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177c268-2b42-47c7-bdec-0a3c5ee2448d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:46.022535 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:46.057835 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152a651-1078-426e-b5dc-1724eabdf0c9 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:46.102552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 66/127] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:07:45 2022] PUT /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc/users/58c09285a32c4e3886cdd7b86f345a0c/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:46.103162 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177c268-2b42-47c7-bdec-0a3c5ee2448d None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:46.127653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea89bb-c295-42c5-b5d6-e9dca392d785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:46.128337 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea89bb-c295-42c5-b5d6-e9dca392d785 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:46.128626 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea89bb-c295-42c5-b5d6-e9dca392d785 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:46.128940 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea89bb-c295-42c5-b5d6-e9dca392d785 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:46.130499 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea89bb-c295-42c5-b5d6-e9dca392d785 None admin] RBAC: Adding query filter params (role.id=0503208661c44c4f8f52e30b4f21fff7, scope.project.id=1fcdd13492724455ae65d6e80d2f41bc, user.id=b04f44fc2e374957b021d2b4110e2056) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:46.130841 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea89bb-c295-42c5-b5d6-e9dca392d785 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:46.136506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea89bb-c295-42c5-b5d6-e9dca392d785 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:46.151102 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 62/128] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:07:45 2022] GET /identity/v3/role_assignments?user.id=b04f44fc2e374957b021d2b4110e2056&scope.project.id=1fcdd13492724455ae65d6e80d2f41bc&role.id=0503208661c44c4f8f52e30b4f21fff7 => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:46.151102 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea89bb-c295-42c5-b5d6-e9dca392d785 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:46.342273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177c268-2b42-47c7-bdec-0a3c5ee2448d None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:46.359277 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:46.369799 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea89bb-c295-42c5-b5d6-e9dca392d785 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:46.417793 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32149ed0-1572-428c-924f-032b2a8addfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:46.419511 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32149ed0-1572-428c-924f-032b2a8addfb None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:46.420189 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32149ed0-1572-428c-924f-032b2a8addfb None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:46.420823 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32149ed0-1572-428c-924f-032b2a8addfb None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:46.422221 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32149ed0-1572-428c-924f-032b2a8addfb None admin] RBAC: Adding query filter params (name=member) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:46.423152 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32149ed0-1572-428c-924f-032b2a8addfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:46.426693 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32149ed0-1572-428c-924f-032b2a8addfb None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:46.443799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 67/129] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:07:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:46.445672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7512b3a-19ba-47b5-9635-d5ac37c52b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:46.446443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7512b3a-19ba-47b5-9635-d5ac37c52b63 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:46.446824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7512b3a-19ba-47b5-9635-d5ac37c52b63 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:46.447309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb702971-f2c9-4ad7-bd40-42ec11f59dfd None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:46.448008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb702971-f2c9-4ad7-bd40-42ec11f59dfd None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:46.448679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7512b3a-19ba-47b5-9635-d5ac37c52b63 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:46.449275 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb702971-f2c9-4ad7-bd40-42ec11f59dfd None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:46.452902 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 68/130] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:46.454006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb702971-f2c9-4ad7-bd40-42ec11f59dfd None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:46.468222 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7512b3a-19ba-47b5-9635-d5ac37c52b63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:46.474192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-f7512b3a-19ba-47b5-9635-d5ac37c52b63 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:07:46.474192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:46.474192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:46.474192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:46.474192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:46.474192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:46.474192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:46.474192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:46.474192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:46.474192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:46.474192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:46.474192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:46.474192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:46.474192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:46.474192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:46.474192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:46.475968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:46.475968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:46.475968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:46.475968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:46.475968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:46.475968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:46.475968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:46.475968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:46.475968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:46.475968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:46.475968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:46.475968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:46.475968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:46.475968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:46.475968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:46.475968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:46.477453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:46.477453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:46.477453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:46.477453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:46.477453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:46.477453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:46.477453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:46.477453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:46.477453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:46.477453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:46.477453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:07:46.477453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:07:46.480851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 63/131] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:07:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:07:46.482196 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6607f-18c0-4b4c-b90c-06b1f0c8b512 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:46.482483 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6607f-18c0-4b4c-b90c-06b1f0c8b512 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:46.482763 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6607f-18c0-4b4c-b90c-06b1f0c8b512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:46.497965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:46.525141 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-60b6607f-18c0-4b4c-b90c-06b1f0c8b512 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:46.527796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-60b6607f-18c0-4b4c-b90c-06b1f0c8b512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:46.661772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb702971-f2c9-4ad7-bd40-42ec11f59dfd None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:46.691631 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 64/132] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:46.693093 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6607f-18c0-4b4c-b90c-06b1f0c8b512 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:46.768209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9309bd-24e3-4772-8b46-0509574a443d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:46.769266 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9309bd-24e3-4772-8b46-0509574a443d None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:46.770381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9309bd-24e3-4772-8b46-0509574a443d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:46.770856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9309bd-24e3-4772-8b46-0509574a443d None admin] PATH_INFO: `/v3/users/58c09285a32c4e3886cdd7b86f345a0c` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:46.774110 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9309bd-24e3-4772-8b46-0509574a443d None admin] RBAC: Authorizing `identity:get_user(user_id=58c09285a32c4e3886cdd7b86f345a0c)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:46.775902 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9309bd-24e3-4772-8b46-0509574a443d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:46.781059 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 69/133] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:46 2022] GET /identity/v3/users/58c09285a32c4e3886cdd7b86f345a0c => generated 275 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:46.786516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9309bd-24e3-4772-8b46-0509574a443d None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:46.846606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:46.931246 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6607f-18c0-4b4c-b90c-06b1f0c8b512 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:47.053629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9309bd-24e3-4772-8b46-0509574a443d None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:47.056379 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419a188-a86e-47d8-a161-644ef2873887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:47.057492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419a188-a86e-47d8-a161-644ef2873887 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:47.058002 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419a188-a86e-47d8-a161-644ef2873887 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:47.058553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419a188-a86e-47d8-a161-644ef2873887 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:47.059852 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2419a188-a86e-47d8-a161-644ef2873887 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:47.060344 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2419a188-a86e-47d8-a161-644ef2873887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:47.063044 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2419a188-a86e-47d8-a161-644ef2873887 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:47.065757 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:47.089235 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 65/134] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:07:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:47.105516 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419a188-a86e-47d8-a161-644ef2873887 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:47.159828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1e725-60dc-4814-b198-9c8ce6e0c948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:47.161373 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1e725-60dc-4814-b198-9c8ce6e0c948 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:47.161670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1e725-60dc-4814-b198-9c8ce6e0c948 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:47.161950 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1e725-60dc-4814-b198-9c8ce6e0c948 None admin] PATH_INFO: `/v3/projects/a4400166f2f64a2a9b6a52c84af30069` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:47.163407 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1e725-60dc-4814-b198-9c8ce6e0c948 None admin] RBAC: Authorizing `identity:get_project(project_id=a4400166f2f64a2a9b6a52c84af30069)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:47.165030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d1e725-60dc-4814-b198-9c8ce6e0c948 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:47.167141 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 70/135] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:07:46 2022] GET /identity/v3/projects/a4400166f2f64a2a9b6a52c84af30069 => generated 312 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:47.167798 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1e725-60dc-4814-b198-9c8ce6e0c948 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:47.330553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419a188-a86e-47d8-a161-644ef2873887 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:47.383554 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:47.397111 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d1e725-60dc-4814-b198-9c8ce6e0c948 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:47.452539 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b39a0-f5d7-471b-be28-5fbdf547ab29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:47.453584 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b39a0-f5d7-471b-be28-5fbdf547ab29 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:47.454075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b39a0-f5d7-471b-be28-5fbdf547ab29 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:47.454544 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b39a0-f5d7-471b-be28-5fbdf547ab29 None admin] PATH_INFO: `/v3/users/58c09285a32c4e3886cdd7b86f345a0c` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:47.456001 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b39a0-f5d7-471b-be28-5fbdf547ab29 None admin] RBAC: Authorizing `identity:get_user(user_id=58c09285a32c4e3886cdd7b86f345a0c)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:47.460075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b39a0-f5d7-471b-be28-5fbdf547ab29 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:47.462644 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 66/136] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:47 2022] GET /identity/v3/users/58c09285a32c4e3886cdd7b86f345a0c => generated 275 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:47.463940 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b39a0-f5d7-471b-be28-5fbdf547ab29 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:47.510757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b8c03-4e59-4a20-9e2a-96da591cf28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:47.512244 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b8c03-4e59-4a20-9e2a-96da591cf28c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:47.512729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b8c03-4e59-4a20-9e2a-96da591cf28c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:47.513235 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b8c03-4e59-4a20-9e2a-96da591cf28c None admin] PATH_INFO: `/v3/domains/default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:47.523588 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b8c03-4e59-4a20-9e2a-96da591cf28c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:47.532415 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b8c03-4e59-4a20-9e2a-96da591cf28c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:47.533909 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 71/137] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:07:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:47.537600 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b8c03-4e59-4a20-9e2a-96da591cf28c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:47.643496 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:47.731053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b39a0-f5d7-471b-be28-5fbdf547ab29 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:47.770670 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555b6cc-651d-40d7-96d2-1ce7a637015f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:47.771358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555b6cc-651d-40d7-96d2-1ce7a637015f None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:47.771651 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555b6cc-651d-40d7-96d2-1ce7a637015f None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:47.771990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555b6cc-651d-40d7-96d2-1ce7a637015f None admin] PATH_INFO: `/v3/projects/a4400166f2f64a2a9b6a52c84af30069/users/58c09285a32c4e3886cdd7b86f345a0c/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:47.774105 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555b6cc-651d-40d7-96d2-1ce7a637015f None admin] RBAC: Authorizing `identity:create_grant(project_id=a4400166f2f64a2a9b6a52c84af30069, user_id=58c09285a32c4e3886cdd7b86f345a0c, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:47.775782 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555b6cc-651d-40d7-96d2-1ce7a637015f None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:47.776119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b8c03-4e59-4a20-9e2a-96da591cf28c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:47.825818 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25c692-bbd7-4ac0-9848-85ef2ba4c313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:47.826860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25c692-bbd7-4ac0-9848-85ef2ba4c313 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:47.827403 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25c692-bbd7-4ac0-9848-85ef2ba4c313 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:47.827892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25c692-bbd7-4ac0-9848-85ef2ba4c313 None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:47.829661 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25c692-bbd7-4ac0-9848-85ef2ba4c313 None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:47.831566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25c692-bbd7-4ac0-9848-85ef2ba4c313 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:47.834706 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 72/138] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:07:47 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:47.835959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25c692-bbd7-4ac0-9848-85ef2ba4c313 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:47.866773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 67/139] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:07:47 2022] PUT /identity/v3/projects/a4400166f2f64a2a9b6a52c84af30069/users/58c09285a32c4e3886cdd7b86f345a0c/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:47.871431 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555b6cc-651d-40d7-96d2-1ce7a637015f None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:47.876952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:47.966620 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25c692-bbd7-4ac0-9848-85ef2ba4c313 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:48.011010 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050fe5b-25a2-4542-b932-da3b9592e426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:48.011010 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050fe5b-25a2-4542-b932-da3b9592e426 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:48.011010 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050fe5b-25a2-4542-b932-da3b9592e426 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:48.011010 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050fe5b-25a2-4542-b932-da3b9592e426 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:48.011716 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050fe5b-25a2-4542-b932-da3b9592e426 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:48.012799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050fe5b-25a2-4542-b932-da3b9592e426 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:48.105120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 73/140] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:07:47 2022] POST /identity/v3/users => generated 283 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:48.120969 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555b6cc-651d-40d7-96d2-1ce7a637015f None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:48.153651 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:48.168535 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb8a14-117f-429c-ad40-ad6c7ac35b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:48.169556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb8a14-117f-429c-ad40-ad6c7ac35b43 None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:48.170002 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb8a14-117f-429c-ad40-ad6c7ac35b43 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:48.170535 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb8a14-117f-429c-ad40-ad6c7ac35b43 None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc/users/58c09285a32c4e3886cdd7b86f345a0c/roles/fc0a7877dd0f4903a2a6fcc01a829c6a` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:48.173450 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb8a14-117f-429c-ad40-ad6c7ac35b43 None admin] RBAC: Authorizing `identity:create_grant(project_id=1fcdd13492724455ae65d6e80d2f41bc, user_id=58c09285a32c4e3886cdd7b86f345a0c, role_id=fc0a7877dd0f4903a2a6fcc01a829c6a)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:48.175245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb8a14-117f-429c-ad40-ad6c7ac35b43 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:48.250742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 68/141] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:07:47 2022] PUT /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc/users/58c09285a32c4e3886cdd7b86f345a0c/roles/fc0a7877dd0f4903a2a6fcc01a829c6a => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:48.898391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f12a2-5f61-4f75-8b4b-a3a0f0072c48 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:48.898923 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f12a2-5f61-4f75-8b4b-a3a0f0072c48 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:48.898923 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f12a2-5f61-4f75-8b4b-a3a0f0072c48 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:48.900968 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 74/142] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:48.907110 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96501eda-d71c-4577-ace5-2986592b976a None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:48.907455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96501eda-d71c-4577-ace5-2986592b976a None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:48.907717 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96501eda-d71c-4577-ace5-2986592b976a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:48.940198 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-96501eda-d71c-4577-ace5-2986592b976a None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:48.942254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-96501eda-d71c-4577-ace5-2986592b976a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:49.045502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 69/143] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:49.989057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f12a2-5f61-4f75-8b4b-a3a0f0072c48 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:50.118724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:50.137604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbeeab6-893b-47cb-b087-db8320dcc54c None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:50.138129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbeeab6-893b-47cb-b087-db8320dcc54c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:50.138361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbeeab6-893b-47cb-b087-db8320dcc54c None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:50.140034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 70/144] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:50.142610 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f12a2-5f61-4f75-8b4b-a3a0f0072c48 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:50.148908 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcca1a-ef0c-4bb5-9a5e-dbbc152962e7 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:50.149200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcca1a-ef0c-4bb5-9a5e-dbbc152962e7 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:50.149473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcca1a-ef0c-4bb5-9a5e-dbbc152962e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:50.185766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-3afcca1a-ef0c-4bb5-9a5e-dbbc152962e7 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:50.187230 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-3afcca1a-ef0c-4bb5-9a5e-dbbc152962e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:50.220746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f7480-a90e-425d-8b4b-676fae21a7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:50.221428 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f7480-a90e-425d-8b4b-676fae21a7fd None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:50.221711 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f7480-a90e-425d-8b4b-676fae21a7fd None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:50.221988 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f7480-a90e-425d-8b4b-676fae21a7fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:50.249132 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f7480-a90e-425d-8b4b-676fae21a7fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:50.255027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-b30f7480-a90e-425d-8b4b-676fae21a7fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:07:50.255027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:50.255027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:50.255027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:50.255027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:50.255027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:50.255027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:50.255027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:50.255027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:50.255027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:50.255027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:50.255027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:50.255027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:50.255027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:50.255027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:50.255027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:50.256289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:50.256289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:50.256289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:50.256289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:50.256289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:50.256289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:50.256289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:50.256289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:50.256289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:50.256289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:50.256289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:50.256289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:50.256289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:50.256289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:50.256289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:50.256289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:50.257057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:50.257057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:50.257057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:50.257057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:50.257057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:50.257057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:50.257057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:50.257057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:50.257057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:50.257057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:50.257057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:07:50.257057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:07:50.263520 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 75/145] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:07:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:07:50.271463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f7480-a90e-425d-8b4b-676fae21a7fd None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:50.413770 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 71/146] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:07:50.534437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f7480-a90e-425d-8b4b-676fae21a7fd None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:50.640239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5cece-4931-4fe8-93fd-9fc58c51e125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:50.641330 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5cece-4931-4fe8-93fd-9fc58c51e125 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:50.641822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5cece-4931-4fe8-93fd-9fc58c51e125 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:50.642307 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5cece-4931-4fe8-93fd-9fc58c51e125 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:50.643659 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5cece-4931-4fe8-93fd-9fc58c51e125 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:50.644211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5cece-4931-4fe8-93fd-9fc58c51e125 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:50.647724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5cece-4931-4fe8-93fd-9fc58c51e125 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:50.669792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 76/147] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:07:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:50.679575 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcca1a-ef0c-4bb5-9a5e-dbbc152962e7 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:50.753776 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c143a-039a-456b-a0b0-cbaf2bbf11e7 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:50.755269 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c143a-039a-456b-a0b0-cbaf2bbf11e7 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:50.755269 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c143a-039a-456b-a0b0-cbaf2bbf11e7 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:50.758293 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 77/148] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:50.767961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e206c8-d981-4a6a-9b8b-600d87a2f95b None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:50.768350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e206c8-d981-4a6a-9b8b-600d87a2f95b None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:50.768645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e206c8-d981-4a6a-9b8b-600d87a2f95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:50.795657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:50.812816 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-91e206c8-d981-4a6a-9b8b-600d87a2f95b None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:50.814879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-91e206c8-d981-4a6a-9b8b-600d87a2f95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:50.847599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:51.020298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcca1a-ef0c-4bb5-9a5e-dbbc152962e7 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:51.082017 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 78/149] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:51.101135 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd579d-7a0a-4e1b-abff-f27c482b3dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:51.102168 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd579d-7a0a-4e1b-abff-f27c482b3dc4 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:51.102648 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd579d-7a0a-4e1b-abff-f27c482b3dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:51.103160 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd579d-7a0a-4e1b-abff-f27c482b3dc4 None admin] PATH_INFO: `/v3/users/7a8e484a77ce40f880d21a33a930f0be` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:51.107852 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd579d-7a0a-4e1b-abff-f27c482b3dc4 None admin] RBAC: Authorizing `identity:get_user(user_id=7a8e484a77ce40f880d21a33a930f0be)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:51.109725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbd579d-7a0a-4e1b-abff-f27c482b3dc4 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:51.111719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 72/150] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:50 2022] GET /identity/v3/users/7a8e484a77ce40f880d21a33a930f0be => generated 247 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:51.119742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e206c8-d981-4a6a-9b8b-600d87a2f95b None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:51.299811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e206c8-d981-4a6a-9b8b-600d87a2f95b None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:51.409395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478b04b-47db-4398-8d56-f8d58f86ba1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:51.410926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478b04b-47db-4398-8d56-f8d58f86ba1c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:51.411818 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478b04b-47db-4398-8d56-f8d58f86ba1c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:51.412570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478b04b-47db-4398-8d56-f8d58f86ba1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:51.423440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:51.560005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478b04b-47db-4398-8d56-f8d58f86ba1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:51.562245 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478b04b-47db-4398-8d56-f8d58f86ba1c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:51.566806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd579d-7a0a-4e1b-abff-f27c482b3dc4 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:51.797759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 79/151] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:07:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 678 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:07:51.806612 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478b04b-47db-4398-8d56-f8d58f86ba1c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:51.881961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbd579d-7a0a-4e1b-abff-f27c482b3dc4 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:51.943406 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:51.945562 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:51.981286 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb173231-0e14-4f8a-9ff8-ab3a64568e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:51.981973 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb173231-0e14-4f8a-9ff8-ab3a64568e86 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:51.982258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb173231-0e14-4f8a-9ff8-ab3a64568e86 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:51.982537 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb173231-0e14-4f8a-9ff8-ab3a64568e86 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:52.008340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb173231-0e14-4f8a-9ff8-ab3a64568e86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:52.014559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-bb173231-0e14-4f8a-9ff8-ab3a64568e86 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:07:52.014559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:52.014559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:52.014559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:52.014559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:52.014559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:52.014559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:52.014559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:52.014559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:52.014559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:52.014559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:52.014559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:52.014559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:52.014559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:52.014559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:52.014559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:52.016217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:52.016217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:52.016217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:52.016217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:52.016217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:52.016217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:52.016217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:52.016217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:52.016217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:52.016217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:52.016217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:52.016217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:52.016217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:52.016217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:52.016217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:52.016217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:52.017697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:52.017697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:52.017697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:52.017697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:52.017697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:52.017697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:52.017697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:52.017697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:52.017697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:52.017697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:52.017697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:07:52.017697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:07:52.021070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 73/152] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:07:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 455 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:07:52.031304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb173231-0e14-4f8a-9ff8-ab3a64568e86 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:52.185602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478b04b-47db-4398-8d56-f8d58f86ba1c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:52.307018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373892e-f0c9-4b40-abc8-afae7d729f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:52.307018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373892e-f0c9-4b40-abc8-afae7d729f4e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:52.307018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373892e-f0c9-4b40-abc8-afae7d729f4e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:52.307018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373892e-f0c9-4b40-abc8-afae7d729f4e None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:52.312760 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373892e-f0c9-4b40-abc8-afae7d729f4e None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:52.315647 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373892e-f0c9-4b40-abc8-afae7d729f4e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:52.318885 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 80/153] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:07:51 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 513 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:52.320471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373892e-f0c9-4b40-abc8-afae7d729f4e None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:52.342413 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb173231-0e14-4f8a-9ff8-ab3a64568e86 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:52.459770 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3cac99-51bc-4261-a3cc-e779424cc310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:52.460615 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cac99-51bc-4261-a3cc-e779424cc310 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:52.461039 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cac99-51bc-4261-a3cc-e779424cc310 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:52.461462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cac99-51bc-4261-a3cc-e779424cc310 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:52.462577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3cac99-51bc-4261-a3cc-e779424cc310 None admin] RBAC: Adding query filter params (name=reader) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:52.467100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3cac99-51bc-4261-a3cc-e779424cc310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:52.467100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3cac99-51bc-4261-a3cc-e779424cc310 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:52.492119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 74/154] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:07:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 461 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:52.493775 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3cac99-51bc-4261-a3cc-e779424cc310 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:52.614036 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:52.693402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373892e-f0c9-4b40-abc8-afae7d729f4e None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:52.704347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:52.811923 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422f7fa-0142-4f49-a294-74d43f11a2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:52.813371 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422f7fa-0142-4f49-a294-74d43f11a2e9 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:52.814196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422f7fa-0142-4f49-a294-74d43f11a2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:52.814895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422f7fa-0142-4f49-a294-74d43f11a2e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:52.835694 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3cac99-51bc-4261-a3cc-e779424cc310 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:52.848699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422f7fa-0142-4f49-a294-74d43f11a2e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:52.854089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-d422f7fa-0142-4f49-a294-74d43f11a2e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:07:52.854089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:52.854089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:52.854089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:52.854089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:52.854089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:52.854089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:52.854089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:52.854089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:52.854089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:52.854089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:52.854089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:52.854089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:52.854089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:52.854089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:52.854089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:52.855197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:52.855197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:52.855197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:52.855197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:52.855197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:52.855197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:52.855197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:52.855197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:52.855197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:52.855197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:52.855197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:52.855197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:52.855197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:52.855197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:52.855197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:52.855197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:52.856179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:52.856179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:52.856179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:52.856179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:52.856179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:52.856179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:52.856179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:52.856179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:52.856179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:52.856179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:52.856179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:07:52.856179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:07:52.858474 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 81/155] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:07:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 539 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:07:52.859542 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422f7fa-0142-4f49-a294-74d43f11a2e9 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:52.961203 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff2baa-ed68-4708-bdf3-8c948120ce4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:52.962313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff2baa-ed68-4708-bdf3-8c948120ce4d None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:52.962818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff2baa-ed68-4708-bdf3-8c948120ce4d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:52.963334 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff2baa-ed68-4708-bdf3-8c948120ce4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:52.964770 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff2baa-ed68-4708-bdf3-8c948120ce4d None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=1fcdd13492724455ae65d6e80d2f41bc, user.id=7a8e484a77ce40f880d21a33a930f0be) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:52.965372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff2baa-ed68-4708-bdf3-8c948120ce4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:52.968219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff2baa-ed68-4708-bdf3-8c948120ce4d None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:52.992817 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 75/156] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:07:52 2022] GET /identity/v3/role_assignments?user.id=7a8e484a77ce40f880d21a33a930f0be&scope.project.id=1fcdd13492724455ae65d6e80d2f41bc&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 608 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:52.994341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff2baa-ed68-4708-bdf3-8c948120ce4d None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:53.102508 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422f7fa-0142-4f49-a294-74d43f11a2e9 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:53.174114 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca085ae-7a5b-4313-8450-366217559433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:53.176298 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca085ae-7a5b-4313-8450-366217559433 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:53.177055 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca085ae-7a5b-4313-8450-366217559433 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:53.177843 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca085ae-7a5b-4313-8450-366217559433 None admin] PATH_INFO: `/v3/users/b04f44fc2e374957b021d2b4110e2056` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:53.182481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca085ae-7a5b-4313-8450-366217559433 None admin] RBAC: Authorizing `identity:get_user(user_id=b04f44fc2e374957b021d2b4110e2056)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:53.185489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca085ae-7a5b-4313-8450-366217559433 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:53.189008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 82/157] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:52 2022] GET /identity/v3/users/b04f44fc2e374957b021d2b4110e2056 => generated 289 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:53.195425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe53bcb-367b-45ee-80cc-b0a3ec4825a9 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:53.196324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe53bcb-367b-45ee-80cc-b0a3ec4825a9 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:53.196797 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe53bcb-367b-45ee-80cc-b0a3ec4825a9 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:53.198781 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 83/158] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:53.202566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe53bcb-367b-45ee-80cc-b0a3ec4825a9 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:53.277298 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:53.377953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:53.397195 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff2baa-ed68-4708-bdf3-8c948120ce4d None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:53.448656 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe53bcb-367b-45ee-80cc-b0a3ec4825a9 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:53.520695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85f6cb-d44c-4310-b7eb-a6c41ce90069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:53.521627 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85f6cb-d44c-4310-b7eb-a6c41ce90069 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:53.521915 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85f6cb-d44c-4310-b7eb-a6c41ce90069 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:53.522188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85f6cb-d44c-4310-b7eb-a6c41ce90069 None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:53.523678 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85f6cb-d44c-4310-b7eb-a6c41ce90069 None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:53.528304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dae889-e5e8-485f-9902-4bbbce2d8c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:53.529145 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dae889-e5e8-485f-9902-4bbbce2d8c99 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:53.529638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dae889-e5e8-485f-9902-4bbbce2d8c99 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:53.530110 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dae889-e5e8-485f-9902-4bbbce2d8c99 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:53.531043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85f6cb-d44c-4310-b7eb-a6c41ce90069 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:53.531493 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dae889-e5e8-485f-9902-4bbbce2d8c99 None admin] RBAC: Adding query filter params (name=member) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:53.532026 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dae889-e5e8-485f-9902-4bbbce2d8c99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:53.532969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 84/159] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:07:53 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:07:53.534552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f648685-c74b-4398-9232-45a287c5f80a None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:53.534752 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dae889-e5e8-485f-9902-4bbbce2d8c99 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:53.535577 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f648685-c74b-4398-9232-45a287c5f80a None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:53.535911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f648685-c74b-4398-9232-45a287c5f80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:53.558564 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 76/160] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:07:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 565 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:53.559620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dae889-e5e8-485f-9902-4bbbce2d8c99 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:53.561715 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-2f648685-c74b-4398-9232-45a287c5f80a None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:53.567719 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-2f648685-c74b-4398-9232-45a287c5f80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:53.739839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 85/161] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:53.740639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f648685-c74b-4398-9232-45a287c5f80a None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:53.807801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dae889-e5e8-485f-9902-4bbbce2d8c99 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:53.808588 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:53.864742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ac066-dbb8-4941-9861-6c5a9ac10bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:53.865476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ac066-dbb8-4941-9861-6c5a9ac10bb8 None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:53.865770 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ac066-dbb8-4941-9861-6c5a9ac10bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:53.866081 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ac066-dbb8-4941-9861-6c5a9ac10bb8 None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc/users/b04f44fc2e374957b021d2b4110e2056/roles/0503208661c44c4f8f52e30b4f21fff7` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:53.868730 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ac066-dbb8-4941-9861-6c5a9ac10bb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=1fcdd13492724455ae65d6e80d2f41bc, user_id=b04f44fc2e374957b021d2b4110e2056, role_id=0503208661c44c4f8f52e30b4f21fff7)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:53.871737 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ac066-dbb8-4941-9861-6c5a9ac10bb8 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:53.955456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f648685-c74b-4398-9232-45a287c5f80a None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:53.965369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 77/162] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:07:53 2022] PUT /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc/users/b04f44fc2e374957b021d2b4110e2056/roles/0503208661c44c4f8f52e30b4f21fff7 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:07:54.013771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bdfa66-1422-4b25-9996-4957d90a34f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:54.014348 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdfa66-1422-4b25-9996-4957d90a34f8 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:54.014629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdfa66-1422-4b25-9996-4957d90a34f8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:54.014953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdfa66-1422-4b25-9996-4957d90a34f8 None admin] PATH_INFO: `/v3/users/58c09285a32c4e3886cdd7b86f345a0c` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:54.016267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bdfa66-1422-4b25-9996-4957d90a34f8 None admin] RBAC: Authorizing `identity:get_user(user_id=58c09285a32c4e3886cdd7b86f345a0c)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:54.017740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bdfa66-1422-4b25-9996-4957d90a34f8 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:54.020429 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 86/163] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:53 2022] GET /identity/v3/users/58c09285a32c4e3886cdd7b86f345a0c => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:54.026670 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ac066-dbb8-4941-9861-6c5a9ac10bb8 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:54.073662 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:54.233475 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ac066-dbb8-4941-9861-6c5a9ac10bb8 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:54.308179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af79fc5-9e3c-4dfb-b643-da5b42223e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:54.308951 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af79fc5-9e3c-4dfb-b643-da5b42223e7f None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:54.309444 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af79fc5-9e3c-4dfb-b643-da5b42223e7f None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:54.309921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af79fc5-9e3c-4dfb-b643-da5b42223e7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:54.401222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15556087-b3be-4d1a-b0b1-bd31f12c35a2 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:54.402660 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15556087-b3be-4d1a-b0b1-bd31f12c35a2 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:54.403198 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15556087-b3be-4d1a-b0b1-bd31f12c35a2 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:54.405395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 87/164] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:07:54.413869 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048303db-1d08-4640-9f8c-3270f3179f64 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:54.414327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048303db-1d08-4640-9f8c-3270f3179f64 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:54.414767 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048303db-1d08-4640-9f8c-3270f3179f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:54.465328 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-048303db-1d08-4640-9f8c-3270f3179f64 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:54.467920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-048303db-1d08-4640-9f8c-3270f3179f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:54.496272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af79fc5-9e3c-4dfb-b643-da5b42223e7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:54.498103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af79fc5-9e3c-4dfb-b643-da5b42223e7f None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:54.552632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:54.689666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 88/165] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:54.690539 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048303db-1d08-4640-9f8c-3270f3179f64 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:54.716281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 78/166] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:07:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:07:54.725958 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af79fc5-9e3c-4dfb-b643-da5b42223e7f None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:54.770574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:55.027049 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af79fc5-9e3c-4dfb-b643-da5b42223e7f None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:55.057609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048303db-1d08-4640-9f8c-3270f3179f64 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:55.083097 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:55.115834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01dd22-188c-4873-9be1-856bbd574718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:55.117057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01dd22-188c-4873-9be1-856bbd574718 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:55.122221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01dd22-188c-4873-9be1-856bbd574718 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:55.122921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01dd22-188c-4873-9be1-856bbd574718 None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:55.125276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b01dd22-188c-4873-9be1-856bbd574718 None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:55.127058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b01dd22-188c-4873-9be1-856bbd574718 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:55.130086 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 79/167] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:07:54 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:55.135013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502db35f-82b2-4b91-ad98-532516f04546 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:55.135013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502db35f-82b2-4b91-ad98-532516f04546 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:55.135013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502db35f-82b2-4b91-ad98-532516f04546 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:55.141732 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 80/168] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:55 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:55.142748 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502db35f-82b2-4b91-ad98-532516f04546 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:55.163388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc0028-3e1b-4c7d-86ff-9cc33449d55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:55.164766 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc0028-3e1b-4c7d-86ff-9cc33449d55c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:55.165470 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc0028-3e1b-4c7d-86ff-9cc33449d55c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:55.166261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc0028-3e1b-4c7d-86ff-9cc33449d55c None admin] PATH_INFO: `/v3/roles/member` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:55.196300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc0028-3e1b-4c7d-86ff-9cc33449d55c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:55.213051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-0cbc0028-3e1b-4c7d-86ff-9cc33449d55c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:07:55.213051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:55.213051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:55.213051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:55.213051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:55.213051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:55.213051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:55.213051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:55.213051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:55.213051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:55.213051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:55.213051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:55.213051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:55.213051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:55.213051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:55.213051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:55.215574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:55.215574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:55.215574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:55.215574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:55.215574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:55.215574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:55.215574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:55.215574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:55.215574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:55.215574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:55.215574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:55.215574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:55.215574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:55.215574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:55.215574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:55.215574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:55.217191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:55.217191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:55.217191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:55.217191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:55.217191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:55.217191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:55.217191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:55.217191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:55.217191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:55.217191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:55.217191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:07:55.217191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:07:55.217191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 89/169] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:07:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 525 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:07:55.219677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07875b3-2ba0-433b-99f3-20aca34e09f9 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:55.219677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07875b3-2ba0-433b-99f3-20aca34e09f9 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:55.219677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07875b3-2ba0-433b-99f3-20aca34e09f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:55.277467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-c07875b3-2ba0-433b-99f3-20aca34e09f9 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:55.279778 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c07875b3-2ba0-433b-99f3-20aca34e09f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:55.475003 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502db35f-82b2-4b91-ad98-532516f04546 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:55.494162 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 90/170] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:55.495980 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07875b3-2ba0-433b-99f3-20aca34e09f9 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:55.558226 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a02b9d-20eb-465d-bc90-ffcbc47759e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:55.568950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a02b9d-20eb-465d-bc90-ffcbc47759e4 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:55.569506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a02b9d-20eb-465d-bc90-ffcbc47759e4 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:55.570121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a02b9d-20eb-465d-bc90-ffcbc47759e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:55.571899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a02b9d-20eb-465d-bc90-ffcbc47759e4 None admin] RBAC: Adding query filter params (role.id=bcc52f73cf03442e922a4757908334de, scope.project.id=1fcdd13492724455ae65d6e80d2f41bc, user.id=58c09285a32c4e3886cdd7b86f345a0c) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:55.572483 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a02b9d-20eb-465d-bc90-ffcbc47759e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:55.579025 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a02b9d-20eb-465d-bc90-ffcbc47759e4 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:55.581525 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:55.603018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 81/171] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:07:55 2022] GET /identity/v3/role_assignments?user.id=58c09285a32c4e3886cdd7b86f345a0c&scope.project.id=1fcdd13492724455ae65d6e80d2f41bc&role.id=bcc52f73cf03442e922a4757908334de => generated 608 bytes in 457 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:55.605774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a02b9d-20eb-465d-bc90-ffcbc47759e4 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:55.759665 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:55.867607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07875b3-2ba0-433b-99f3-20aca34e09f9 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:55.871178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a02b9d-20eb-465d-bc90-ffcbc47759e4 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:55.955066 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d591357-8a02-4dd0-8b36-473c4a9feab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:55.956038 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d591357-8a02-4dd0-8b36-473c4a9feab1 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:55.956474 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d591357-8a02-4dd0-8b36-473c4a9feab1 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:55.957416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d591357-8a02-4dd0-8b36-473c4a9feab1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:55.968178 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db39fb-c023-4997-8ed5-2e21a047b88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:55.969304 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db39fb-c023-4997-8ed5-2e21a047b88a None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:55.969760 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db39fb-c023-4997-8ed5-2e21a047b88a None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:55.970314 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db39fb-c023-4997-8ed5-2e21a047b88a None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:55.971574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db39fb-c023-4997-8ed5-2e21a047b88a None admin] RBAC: Adding query filter params (name=member) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:55.972084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db39fb-c023-4997-8ed5-2e21a047b88a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:55.975491 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db39fb-c023-4997-8ed5-2e21a047b88a None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:55.985987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d591357-8a02-4dd0-8b36-473c4a9feab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:55.994582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 91/172] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:07:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 499 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:55.997138 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270a1dc-0171-4dad-8b78-a01873ae230c None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:55.997676 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270a1dc-0171-4dad-8b78-a01873ae230c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:55.998151 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270a1dc-0171-4dad-8b78-a01873ae230c None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:56.000682 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 92/173] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:56.001461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-6d591357-8a02-4dd0-8b36-473c4a9feab1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:07:56.001461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:56.001461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:56.001461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:56.001461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:56.001461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:56.001461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:56.001461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:56.001461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:56.001461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:56.001461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:56.001461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:56.001461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:56.001461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:56.001461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:56.001461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:56.003411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:56.003411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:56.003411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:56.003411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:56.003411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:56.003411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:56.003411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:56.003411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:56.003411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:56.003411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:56.003411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:56.003411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:56.003411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:56.003411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:56.003411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:56.003411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:56.005721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:56.005721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:56.005721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:56.005721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:56.005721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:56.005721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:56.005721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:56.005721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:56.005721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:56.005721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:56.005721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:07:56.005721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:07:56.005721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 82/174] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:07:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:07:56.009565 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947546c-f0fe-4ae3-b1eb-741b945fa64d None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:56.011586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947546c-f0fe-4ae3-b1eb-741b945fa64d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:56.011586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947546c-f0fe-4ae3-b1eb-741b945fa64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:56.013741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d591357-8a02-4dd0-8b36-473c4a9feab1 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:56.052811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-4947546c-f0fe-4ae3-b1eb-741b945fa64d None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:56.054814 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-4947546c-f0fe-4ae3-b1eb-741b945fa64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:56.209939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 93/175] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:56.211753 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947546c-f0fe-4ae3-b1eb-741b945fa64d None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:56.239645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:56.253315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d591357-8a02-4dd0-8b36-473c4a9feab1 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:56.280566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:56.345644 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007221a1-ab02-4659-8153-fa1bf0a97dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:56.346733 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007221a1-ab02-4659-8153-fa1bf0a97dc4 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:56.348492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007221a1-ab02-4659-8153-fa1bf0a97dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:56.348492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007221a1-ab02-4659-8153-fa1bf0a97dc4 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:56.350569 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007221a1-ab02-4659-8153-fa1bf0a97dc4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:56.350924 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007221a1-ab02-4659-8153-fa1bf0a97dc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:56.353769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007221a1-ab02-4659-8153-fa1bf0a97dc4 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:56.367388 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 83/176] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:07:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:56.368914 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d8ddf-15f4-454d-90b4-f8a10805ea5e None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:56.369205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d8ddf-15f4-454d-90b4-f8a10805ea5e None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:56.369486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d8ddf-15f4-454d-90b4-f8a10805ea5e None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:56.377585 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 84/177] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:56.378754 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d8ddf-15f4-454d-90b4-f8a10805ea5e None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:56.461180 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947546c-f0fe-4ae3-b1eb-741b945fa64d None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:56.533720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265347f-045d-484f-96dc-e4570456ce1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:56.535572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265347f-045d-484f-96dc-e4570456ce1e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:56.536216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265347f-045d-484f-96dc-e4570456ce1e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:56.537015 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265347f-045d-484f-96dc-e4570456ce1e None admin] PATH_INFO: `/v3/users/58c09285a32c4e3886cdd7b86f345a0c` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:56.539313 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265347f-045d-484f-96dc-e4570456ce1e None admin] RBAC: Authorizing `identity:get_user(user_id=58c09285a32c4e3886cdd7b86f345a0c)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:56.542458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265347f-045d-484f-96dc-e4570456ce1e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:56.551043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 94/178] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:56 2022] GET /identity/v3/users/58c09285a32c4e3886cdd7b86f345a0c => generated 275 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:56.551043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265347f-045d-484f-96dc-e4570456ce1e None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:56.592204 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d8ddf-15f4-454d-90b4-f8a10805ea5e None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:56.638543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3cbd9c-3957-470c-bb9a-69146beed9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:56.639988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cbd9c-3957-470c-bb9a-69146beed9fd None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:56.640654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cbd9c-3957-470c-bb9a-69146beed9fd None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:56.641136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cbd9c-3957-470c-bb9a-69146beed9fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:56.654937 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3cbd9c-3957-470c-bb9a-69146beed9fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:56.662907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-0a3cbd9c-3957-470c-bb9a-69146beed9fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:07:56.662907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:56.662907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:56.662907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:56.662907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:56.662907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:56.662907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:56.662907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:56.662907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:56.662907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:56.662907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:56.662907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:56.662907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:56.662907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:56.662907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:56.662907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:56.664177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:56.664177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:56.664177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:56.664177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:56.664177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:56.664177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:56.664177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:56.664177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:56.664177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:56.664177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:56.664177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:56.664177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:56.664177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:56.664177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:56.664177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:56.664177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:56.665279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:56.665279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:56.665279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:56.665279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:56.665279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:56.665279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:56.665279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:56.665279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:56.665279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:56.665279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:56.665279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:07:56.665279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:07:56.668397 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 85/179] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:07:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:07:56.670456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258b5a8-d5ce-4ffe-bcb6-1f45b6b629bf None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:56.671302 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258b5a8-d5ce-4ffe-bcb6-1f45b6b629bf None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:56.672066 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258b5a8-d5ce-4ffe-bcb6-1f45b6b629bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:56.697818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-e258b5a8-d5ce-4ffe-bcb6-1f45b6b629bf None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:56.699747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e258b5a8-d5ce-4ffe-bcb6-1f45b6b629bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:56.730423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:56.738932 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:56.773776 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265347f-045d-484f-96dc-e4570456ce1e None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:56.809102 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 86/180] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:56.810924 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258b5a8-d5ce-4ffe-bcb6-1f45b6b629bf None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:56.852740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e20cf1-475a-4f06-b40d-b3a70695fbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:56.854382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e20cf1-475a-4f06-b40d-b3a70695fbca None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:56.855147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e20cf1-475a-4f06-b40d-b3a70695fbca None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:56.855845 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e20cf1-475a-4f06-b40d-b3a70695fbca None admin] PATH_INFO: `/v3/users/58c09285a32c4e3886cdd7b86f345a0c` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:56.858454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e20cf1-475a-4f06-b40d-b3a70695fbca None admin] RBAC: Authorizing `identity:get_user(user_id=58c09285a32c4e3886cdd7b86f345a0c)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:56.861565 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e20cf1-475a-4f06-b40d-b3a70695fbca None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:56.866664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 95/181] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:56 2022] GET /identity/v3/users/58c09285a32c4e3886cdd7b86f345a0c => generated 275 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:56.868717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e20cf1-475a-4f06-b40d-b3a70695fbca None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:56.978288 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258b5a8-d5ce-4ffe-bcb6-1f45b6b629bf None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:57.042833 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ba9c2-fff5-4bd3-ba73-4c25132649cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:57.043886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ba9c2-fff5-4bd3-ba73-4c25132649cd None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:57.044417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ba9c2-fff5-4bd3-ba73-4c25132649cd None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:57.044895 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ba9c2-fff5-4bd3-ba73-4c25132649cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:57.059602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e20cf1-475a-4f06-b40d-b3a70695fbca None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:57.085548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:57.118548 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea7e02-85b9-486b-9c74-143eba0b135b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:57.119247 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea7e02-85b9-486b-9c74-143eba0b135b None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:57.119644 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea7e02-85b9-486b-9c74-143eba0b135b None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:57.120050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea7e02-85b9-486b-9c74-143eba0b135b None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:57.120765 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea7e02-85b9-486b-9c74-143eba0b135b None admin] RBAC: Adding query filter params (name=admin) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:57.121074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea7e02-85b9-486b-9c74-143eba0b135b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:57.123600 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea7e02-85b9-486b-9c74-143eba0b135b None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:57.145573 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 96/182] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:07:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:57.151220 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea7e02-85b9-486b-9c74-143eba0b135b None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:57.166348 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ba9c2-fff5-4bd3-ba73-4c25132649cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:57.169811 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ba9c2-fff5-4bd3-ba73-4c25132649cd None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:57.176889 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:57.354681 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 87/183] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:07:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:07:57.355464 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea7e02-85b9-486b-9c74-143eba0b135b None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:57.357494 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ba9c2-fff5-4bd3-ba73-4c25132649cd None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:57.420152 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff869b-a951-4571-9784-ddd047c5d087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:57.420854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff869b-a951-4571-9784-ddd047c5d087 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:57.421144 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff869b-a951-4571-9784-ddd047c5d087 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:57.421525 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff869b-a951-4571-9784-ddd047c5d087 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:57.436818 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff869b-a951-4571-9784-ddd047c5d087 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:07:57.443387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-d6ff869b-a951-4571-9784-ddd047c5d087 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:07:57.443387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:07:57.443387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:07:57.443387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:07:57.443387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:07:57.443387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:07:57.443387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:07:57.443387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:07:57.443387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:07:57.443387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:07:57.443387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:07:57.443387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:07:57.443387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:07:57.443387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:07:57.443387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:07:57.443387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:07:57.447309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:07:57.447309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:07:57.447309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:07:57.447309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:07:57.447309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:07:57.447309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:07:57.447309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:07:57.447309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:07:57.447309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:07:57.447309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:07:57.447309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:07:57.447309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:07:57.447309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:07:57.447309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:07:57.447309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:07:57.447309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:07:57.448207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:07:57.448207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:07:57.448207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:07:57.448207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:07:57.448207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:07:57.448207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:07:57.448207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:07:57.448207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:07:57.448207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:07:57.448207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:07:57.448207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:07:57.448207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:07:57.448207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 97/184] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:07:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:07:57.448207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff869b-a951-4571-9784-ddd047c5d087 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:57.461797 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:57.537723 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ba9c2-fff5-4bd3-ba73-4c25132649cd None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:57.580887 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:57.597832 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224242a-3856-4f35-b285-6f3e6ef2b9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:57.599231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224242a-3856-4f35-b285-6f3e6ef2b9a9 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:57.599552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224242a-3856-4f35-b285-6f3e6ef2b9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:57.600009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224242a-3856-4f35-b285-6f3e6ef2b9a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:57.641828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff869b-a951-4571-9784-ddd047c5d087 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:57.700586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74777ef-55ac-4f7a-88ec-8f45b755f6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:57.701708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74777ef-55ac-4f7a-88ec-8f45b755f6cf None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:57.702194 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74777ef-55ac-4f7a-88ec-8f45b755f6cf None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:57.702675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74777ef-55ac-4f7a-88ec-8f45b755f6cf None admin] PATH_INFO: `/v3/users/2370ae01914048938a0317d4add8948b` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:57.723881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224242a-3856-4f35-b285-6f3e6ef2b9a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:57.724873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224242a-3856-4f35-b285-6f3e6ef2b9a9 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:57.727593 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74777ef-55ac-4f7a-88ec-8f45b755f6cf None admin] RBAC: Authorizing `identity:get_user(user_id=2370ae01914048938a0317d4add8948b)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:57.729378 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74777ef-55ac-4f7a-88ec-8f45b755f6cf None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:57.731201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 98/185] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:57 2022] GET /identity/v3/users/2370ae01914048938a0317d4add8948b => generated 283 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:57.736410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74777ef-55ac-4f7a-88ec-8f45b755f6cf None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:57.858780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:57.909860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74777ef-55ac-4f7a-88ec-8f45b755f6cf None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:57.929570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 88/186] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:07:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:07:57.929760 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224242a-3856-4f35-b285-6f3e6ef2b9a9 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:57.967586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:57.983828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4881fb-41f0-4bef-a1c0-f234643c2a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:57.984789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4881fb-41f0-4bef-a1c0-f234643c2a51 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:57.985237 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4881fb-41f0-4bef-a1c0-f234643c2a51 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:57.985668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4881fb-41f0-4bef-a1c0-f234643c2a51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:57.987821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4881fb-41f0-4bef-a1c0-f234643c2a51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:57.990962 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4881fb-41f0-4bef-a1c0-f234643c2a51 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:57.999100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 99/187] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:07:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:57.999100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4881fb-41f0-4bef-a1c0-f234643c2a51 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:58.099924 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224242a-3856-4f35-b285-6f3e6ef2b9a9 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:58.155203 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001cd931-0f4b-4c5e-b686-bbd259aecc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:58.197979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cd931-0f4b-4c5e-b686-bbd259aecc88 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:58.197979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cd931-0f4b-4c5e-b686-bbd259aecc88 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:58.197979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cd931-0f4b-4c5e-b686-bbd259aecc88 None admin] PATH_INFO: `/v3/projects/a4400166f2f64a2a9b6a52c84af30069` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:58.197979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001cd931-0f4b-4c5e-b686-bbd259aecc88 None admin] RBAC: Authorizing `identity:get_project(project_id=a4400166f2f64a2a9b6a52c84af30069)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:58.197979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001cd931-0f4b-4c5e-b686-bbd259aecc88 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:58.197979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 89/188] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:07:57 2022] GET /identity/v3/projects/a4400166f2f64a2a9b6a52c84af30069 => generated 312 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:58.198493 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4881fb-41f0-4bef-a1c0-f234643c2a51 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:58.198605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001cd931-0f4b-4c5e-b686-bbd259aecc88 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:58.225141 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:58.227996 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e88223-51e8-4dde-8daf-5a4dcd8687d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:58.229103 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e88223-51e8-4dde-8daf-5a4dcd8687d1 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:58.230364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e88223-51e8-4dde-8daf-5a4dcd8687d1 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:58.230364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e88223-51e8-4dde-8daf-5a4dcd8687d1 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:58.231010 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e88223-51e8-4dde-8daf-5a4dcd8687d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:58.231250 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e88223-51e8-4dde-8daf-5a4dcd8687d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:58.238813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e88223-51e8-4dde-8daf-5a4dcd8687d1 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:58.253244 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 100/189] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:07:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:58.253244 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e88223-51e8-4dde-8daf-5a4dcd8687d1 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:58.351491 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001cd931-0f4b-4c5e-b686-bbd259aecc88 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:58.355001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:58.404688 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922ce1c-8c62-433a-a83f-8da306882afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:58.406518 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922ce1c-8c62-433a-a83f-8da306882afc None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:58.407241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922ce1c-8c62-433a-a83f-8da306882afc None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:58.407653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922ce1c-8c62-433a-a83f-8da306882afc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:58.416565 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e88223-51e8-4dde-8daf-5a4dcd8687d1 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:58.465617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d965693-bc90-47a4-a876-b9776f3db67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:58.466447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d965693-bc90-47a4-a876-b9776f3db67e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:58.466860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d965693-bc90-47a4-a876-b9776f3db67e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:58.467381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d965693-bc90-47a4-a876-b9776f3db67e None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:58.469224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d965693-bc90-47a4-a876-b9776f3db67e None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:58.471390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d965693-bc90-47a4-a876-b9776f3db67e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:58.473555 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 101/190] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:07:58 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:58.474900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d965693-bc90-47a4-a876-b9776f3db67e None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:58.561298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922ce1c-8c62-433a-a83f-8da306882afc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:58.564502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922ce1c-8c62-433a-a83f-8da306882afc None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:58.615555 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:58.637268 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d965693-bc90-47a4-a876-b9776f3db67e None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:58.677528 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:58.697292 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0a6a7-91ca-499f-a66f-d22103072d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:58.698224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0a6a7-91ca-499f-a66f-d22103072d53 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:58.698641 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0a6a7-91ca-499f-a66f-d22103072d53 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:58.699091 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0a6a7-91ca-499f-a66f-d22103072d53 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:58.700175 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0a6a7-91ca-499f-a66f-d22103072d53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:58.702636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0a6a7-91ca-499f-a66f-d22103072d53 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:58.747769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 90/191] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:07:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:07:58.748718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922ce1c-8c62-433a-a83f-8da306882afc None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:58.841672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 102/192] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:07:58 2022] POST /identity/v3/users => generated 297 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:07:58.842276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0a6a7-91ca-499f-a66f-d22103072d53 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:58.923480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922ce1c-8c62-433a-a83f-8da306882afc None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:58.982738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:58.992109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd76ee4-4f4f-4bcc-8660-3b3336668c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:58.993271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd76ee4-4f4f-4bcc-8660-3b3336668c19 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:58.993778 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd76ee4-4f4f-4bcc-8660-3b3336668c19 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:58.994281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd76ee4-4f4f-4bcc-8660-3b3336668c19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:58.995894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd76ee4-4f4f-4bcc-8660-3b3336668c19 None admin] RBAC: Adding query filter params (role.id=bcc52f73cf03442e922a4757908334de, scope.project.id=a4400166f2f64a2a9b6a52c84af30069, user.id=58c09285a32c4e3886cdd7b86f345a0c) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:58.996460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd76ee4-4f4f-4bcc-8660-3b3336668c19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:59.006296 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd76ee4-4f4f-4bcc-8660-3b3336668c19 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:59.013231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 91/193] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:07:58 2022] GET /identity/v3/role_assignments?user.id=58c09285a32c4e3886cdd7b86f345a0c&scope.project.id=a4400166f2f64a2a9b6a52c84af30069&role.id=bcc52f73cf03442e922a4757908334de => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:59.014819 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd76ee4-4f4f-4bcc-8660-3b3336668c19 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:59.092371 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0a6a7-91ca-499f-a66f-d22103072d53 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:59.185916 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2677e46-7f0c-4f00-bd55-0541f102f7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:59.186963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2677e46-7f0c-4f00-bd55-0541f102f7f8 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:59.187472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2677e46-7f0c-4f00-bd55-0541f102f7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:59.187940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2677e46-7f0c-4f00-bd55-0541f102f7f8 None admin] PATH_INFO: `/v3/users/2370ae01914048938a0317d4add8948b` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:59.190128 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2677e46-7f0c-4f00-bd55-0541f102f7f8 None admin] RBAC: Authorizing `identity:get_user(user_id=2370ae01914048938a0317d4add8948b)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:59.193508 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2677e46-7f0c-4f00-bd55-0541f102f7f8 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:59.198005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 103/194] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:07:58 2022] GET /identity/v3/users/2370ae01914048938a0317d4add8948b => generated 283 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:59.199521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2677e46-7f0c-4f00-bd55-0541f102f7f8 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:59.253805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd76ee4-4f4f-4bcc-8660-3b3336668c19 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:59.322508 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:59.327239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da24020-8c1f-417d-9856-72a9b63a51c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:59.328289 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da24020-8c1f-417d-9856-72a9b63a51c3 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:59.328787 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da24020-8c1f-417d-9856-72a9b63a51c3 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:59.329939 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da24020-8c1f-417d-9856-72a9b63a51c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:59.330699 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da24020-8c1f-417d-9856-72a9b63a51c3 None admin] RBAC: Adding query filter params (role.id=fc0a7877dd0f4903a2a6fcc01a829c6a, scope.project.id=1fcdd13492724455ae65d6e80d2f41bc, user.id=58c09285a32c4e3886cdd7b86f345a0c) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:07:59.331160 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da24020-8c1f-417d-9856-72a9b63a51c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:59.333747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da24020-8c1f-417d-9856-72a9b63a51c3 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:59.346814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 92/195] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:07:59 2022] GET /identity/v3/role_assignments?user.id=58c09285a32c4e3886cdd7b86f345a0c&scope.project.id=1fcdd13492724455ae65d6e80d2f41bc&role.id=fc0a7877dd0f4903a2a6fcc01a829c6a => generated 608 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:59.349552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7076e-b083-418d-a2ff-4bbfc95199f7 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:59.350148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7076e-b083-418d-a2ff-4bbfc95199f7 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:59.350616 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7076e-b083-418d-a2ff-4bbfc95199f7 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:59.353581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 93/196] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:07:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:07:59.360647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7076e-b083-418d-a2ff-4bbfc95199f7 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:59.470709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2677e46-7f0c-4f00-bd55-0541f102f7f8 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:59.535994 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:59.576396 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d63bd-5fed-49bd-8e4d-691ed2955f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:59.577449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d63bd-5fed-49bd-8e4d-691ed2955f37 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:59.577939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d63bd-5fed-49bd-8e4d-691ed2955f37 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:59.578422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d63bd-5fed-49bd-8e4d-691ed2955f37 None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:59.580788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d63bd-5fed-49bd-8e4d-691ed2955f37 None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:59.584906 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d63bd-5fed-49bd-8e4d-691ed2955f37 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:59.588726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 104/197] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:07:59 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:07:59.591121 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ab87d-98d2-4c75-916a-fb23af0fe3f9 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:59.591616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ab87d-98d2-4c75-916a-fb23af0fe3f9 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:59.592109 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ab87d-98d2-4c75-916a-fb23af0fe3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:59.627021 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-497ab87d-98d2-4c75-916a-fb23af0fe3f9 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:07:59.631817 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-497ab87d-98d2-4c75-916a-fb23af0fe3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:07:59.648511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7076e-b083-418d-a2ff-4bbfc95199f7 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:07:59.703957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a66c9-127c-4d3f-824e-2edb4b8cb5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:07:59.705041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a66c9-127c-4d3f-824e-2edb4b8cb5b6 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:07:59.705571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a66c9-127c-4d3f-824e-2edb4b8cb5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:07:59.706079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a66c9-127c-4d3f-824e-2edb4b8cb5b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:07:59.780235 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 105/198] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:07:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:07:59.780777 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ab87d-98d2-4c75-916a-fb23af0fe3f9 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:07:59.838059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a66c9-127c-4d3f-824e-2edb4b8cb5b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:07:59.842299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a66c9-127c-4d3f-824e-2edb4b8cb5b6 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:07:59.867426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:59.955213 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:07:59.986238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ab87d-98d2-4c75-916a-fb23af0fe3f9 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:00.042588 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 94/199] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:07:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:08:00.052660 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a66c9-127c-4d3f-824e-2edb4b8cb5b6 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:00.086934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a271035-d116-4455-afc6-a38ba455ecc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:00.088483 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a271035-d116-4455-afc6-a38ba455ecc4 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:00.090674 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a271035-d116-4455-afc6-a38ba455ecc4 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:00.091167 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a271035-d116-4455-afc6-a38ba455ecc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:00.092239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a271035-d116-4455-afc6-a38ba455ecc4 None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=920dfa8ad35b4a429aea367ae3067b8e, user.id=2370ae01914048938a0317d4add8948b) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:00.092915 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a271035-d116-4455-afc6-a38ba455ecc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:00.096797 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a271035-d116-4455-afc6-a38ba455ecc4 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:00.112224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 106/200] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:07:59 2022] GET /identity/v3/role_assignments?user.id=2370ae01914048938a0317d4add8948b&scope.project.id=920dfa8ad35b4a429aea367ae3067b8e&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 259 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:00.332135 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a66c9-127c-4d3f-824e-2edb4b8cb5b6 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:00.400913 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a271035-d116-4455-afc6-a38ba455ecc4 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:00.424641 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6ab8d-5434-4ce1-a0e6-2279cb67643f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:00.427462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6ab8d-5434-4ce1-a0e6-2279cb67643f None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:00.428036 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6ab8d-5434-4ce1-a0e6-2279cb67643f None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:00.428546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6ab8d-5434-4ce1-a0e6-2279cb67643f None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:00.436528 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6ab8d-5434-4ce1-a0e6-2279cb67643f None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:00.439536 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6ab8d-5434-4ce1-a0e6-2279cb67643f None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:00.445119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 95/201] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:00 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:08:00.454628 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6ab8d-5434-4ce1-a0e6-2279cb67643f None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:00.520733 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:00.631075 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a271035-d116-4455-afc6-a38ba455ecc4 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:00.648025 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:00.682007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6ab8d-5434-4ce1-a0e6-2279cb67643f None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:00.685131 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef562e-24a8-4429-9fd9-efbd454b3a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:00.687417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef562e-24a8-4429-9fd9-efbd454b3a7b None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:00.687417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef562e-24a8-4429-9fd9-efbd454b3a7b None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:00.687417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef562e-24a8-4429-9fd9-efbd454b3a7b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:00.714202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef562e-24a8-4429-9fd9-efbd454b3a7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:00.726206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-74ef562e-24a8-4429-9fd9-efbd454b3a7b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:08:00.726206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:00.726206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:00.726206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:00.726206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:00.726206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:00.726206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:00.726206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:00.726206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:00.726206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:00.726206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:00.726206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:00.726206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:00.726206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:00.726206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:00.726206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:00.728043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:00.728043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:00.728043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:00.728043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:00.728043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:00.728043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:00.728043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:00.728043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:00.728043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:00.728043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:00.728043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:00.728043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:00.728043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:00.728043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:00.728043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:00.728043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:00.729349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:00.729349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:00.729349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:00.729349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:00.729349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:00.729349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:00.729349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:00.729349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:00.729349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:00.729349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:00.729349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:08:00.729349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:08:00.729349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 107/202] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:00.737858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef562e-24a8-4429-9fd9-efbd454b3a7b None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:00.756794 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779676d-402e-4f8a-a2e2-d110baa2cf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:00.758177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779676d-402e-4f8a-a2e2-d110baa2cf5a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:00.759043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779676d-402e-4f8a-a2e2-d110baa2cf5a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:00.759832 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779676d-402e-4f8a-a2e2-d110baa2cf5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:00.761692 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779676d-402e-4f8a-a2e2-d110baa2cf5a None admin] RBAC: Adding query filter params (role.id=fc0a7877dd0f4903a2a6fcc01a829c6a, scope.project.id=920dfa8ad35b4a429aea367ae3067b8e, user.id=2370ae01914048938a0317d4add8948b) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:00.762601 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779676d-402e-4f8a-a2e2-d110baa2cf5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:00.765780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779676d-402e-4f8a-a2e2-d110baa2cf5a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:00.777806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 96/203] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:08:00 2022] GET /identity/v3/role_assignments?user.id=2370ae01914048938a0317d4add8948b&scope.project.id=920dfa8ad35b4a429aea367ae3067b8e&role.id=fc0a7877dd0f4903a2a6fcc01a829c6a => generated 259 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:08:00.918811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef562e-24a8-4429-9fd9-efbd454b3a7b None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:00.973540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac59fc6-fb62-4cc2-a6cd-8eddd2b38423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:00.974149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac59fc6-fb62-4cc2-a6cd-8eddd2b38423 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:00.974490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac59fc6-fb62-4cc2-a6cd-8eddd2b38423 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:00.974863 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac59fc6-fb62-4cc2-a6cd-8eddd2b38423 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:00.975796 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac59fc6-fb62-4cc2-a6cd-8eddd2b38423 None admin] RBAC: Adding query filter params (name=reader) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:00.976315 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac59fc6-fb62-4cc2-a6cd-8eddd2b38423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:00.978532 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac59fc6-fb62-4cc2-a6cd-8eddd2b38423 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:00.990760 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 108/204] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:08:01.000246 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779676d-402e-4f8a-a2e2-d110baa2cf5a None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:01.233199 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:01.247190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779676d-402e-4f8a-a2e2-d110baa2cf5a None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:01.336764 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e81b81-d42c-4d66-95f4-57ff3250ea99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:01.337885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e81b81-d42c-4d66-95f4-57ff3250ea99 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:01.338375 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e81b81-d42c-4d66-95f4-57ff3250ea99 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:01.338863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e81b81-d42c-4d66-95f4-57ff3250ea99 None admin] PATH_INFO: `/v3/users/b04f44fc2e374957b021d2b4110e2056` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:01.344505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e81b81-d42c-4d66-95f4-57ff3250ea99 None admin] RBAC: Authorizing `identity:get_user(user_id=b04f44fc2e374957b021d2b4110e2056)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:01.347482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e81b81-d42c-4d66-95f4-57ff3250ea99 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:01.350949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 97/205] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:08:00 2022] GET /identity/v3/users/b04f44fc2e374957b021d2b4110e2056 => generated 289 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:01.360816 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac59fc6-fb62-4cc2-a6cd-8eddd2b38423 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:01.447585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:01.610331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac59fc6-fb62-4cc2-a6cd-8eddd2b38423 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:01.688890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4413738-81df-407d-81f4-45bf6c881dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:01.690072 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4413738-81df-407d-81f4-45bf6c881dce None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:01.690564 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4413738-81df-407d-81f4-45bf6c881dce None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:01.691218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4413738-81df-407d-81f4-45bf6c881dce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:01.849603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4413738-81df-407d-81f4-45bf6c881dce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:01.852526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4413738-81df-407d-81f4-45bf6c881dce None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:01.914391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:02.043206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 109/206] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:08:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:08:02.048176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e81b81-d42c-4d66-95f4-57ff3250ea99 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:02.333050 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e81b81-d42c-4d66-95f4-57ff3250ea99 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:02.413746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbac78-a275-4af6-8d46-593b9026b48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:02.414870 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbac78-a275-4af6-8d46-593b9026b48e None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:02.415405 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbac78-a275-4af6-8d46-593b9026b48e None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:02.415894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbac78-a275-4af6-8d46-593b9026b48e None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:02.417462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbac78-a275-4af6-8d46-593b9026b48e None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:02.419250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbac78-a275-4af6-8d46-593b9026b48e None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:02.421547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 98/207] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:02 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:02.428028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4413738-81df-407d-81f4-45bf6c881dce None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:02.568120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4413738-81df-407d-81f4-45bf6c881dce None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:02.619427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:02.631188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a50db7-b040-4009-9827-d930d997923a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:02.631997 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a50db7-b040-4009-9827-d930d997923a None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:02.638441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a50db7-b040-4009-9827-d930d997923a None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:02.638441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a50db7-b040-4009-9827-d930d997923a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:02.638441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a50db7-b040-4009-9827-d930d997923a None admin] RBAC: Adding query filter params (role.id=0503208661c44c4f8f52e30b4f21fff7, scope.project.id=1fcdd13492724455ae65d6e80d2f41bc, user.id=b04f44fc2e374957b021d2b4110e2056) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:02.638441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a50db7-b040-4009-9827-d930d997923a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:02.638441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a50db7-b040-4009-9827-d930d997923a None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:02.644229 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 110/208] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:08:02 2022] GET /identity/v3/role_assignments?user.id=b04f44fc2e374957b021d2b4110e2056&scope.project.id=1fcdd13492724455ae65d6e80d2f41bc&role.id=0503208661c44c4f8f52e30b4f21fff7 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:03.486241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e8bd9-f707-4ef5-b945-3c413312c2c6 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:03.487325 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e8bd9-f707-4ef5-b945-3c413312c2c6 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:03.489331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e8bd9-f707-4ef5-b945-3c413312c2c6 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:03.491449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 99/209] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:03.497702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6d77a-bbc1-4936-b6b8-191a1dbdbde1 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:03.498734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6d77a-bbc1-4936-b6b8-191a1dbdbde1 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:03.499280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6d77a-bbc1-4936-b6b8-191a1dbdbde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:03.526666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-f4c6d77a-bbc1-4936-b6b8-191a1dbdbde1 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:03.528407 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f4c6d77a-bbc1-4936-b6b8-191a1dbdbde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:03.649781 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 111/210] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:03.656594 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd775e5-d8a4-44ca-bc87-db970ba1b8cd None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:03.657147 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd775e5-d8a4-44ca-bc87-db970ba1b8cd None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:03.657568 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd775e5-d8a4-44ca-bc87-db970ba1b8cd None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:03.660014 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 100/211] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:03.666246 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7183e58-186d-44bb-a4bf-e03b923ceae1 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:03.666624 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7183e58-186d-44bb-a4bf-e03b923ceae1 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:03.667004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7183e58-186d-44bb-a4bf-e03b923ceae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:03.689713 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-f7183e58-186d-44bb-a4bf-e03b923ceae1 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:03.691723 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f7183e58-186d-44bb-a4bf-e03b923ceae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:03.778898 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 112/212] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:03.881672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf203ee-6005-40ca-8d94-b0a25d28babe None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:03.882472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf203ee-6005-40ca-8d94-b0a25d28babe None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:03.883221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf203ee-6005-40ca-8d94-b0a25d28babe None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:03.886143 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 101/213] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:03.896283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdca2c-b882-439a-abe6-37fa49019c06 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:03.897075 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdca2c-b882-439a-abe6-37fa49019c06 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:03.897774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdca2c-b882-439a-abe6-37fa49019c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:03.927404 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-7fcdca2c-b882-439a-abe6-37fa49019c06 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:03.929472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7fcdca2c-b882-439a-abe6-37fa49019c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:04.025766 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:04.058119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 113/214] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:04.099424 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf203ee-6005-40ca-8d94-b0a25d28babe None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:04.104235 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdca2c-b882-439a-abe6-37fa49019c06 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:04.161839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:04.257438 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdca2c-b882-439a-abe6-37fa49019c06 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:04.316084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c41df-1c94-4198-becf-98c32615ecd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:04.317675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c41df-1c94-4198-becf-98c32615ecd9 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:04.318141 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c41df-1c94-4198-becf-98c32615ecd9 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:04.318652 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c41df-1c94-4198-becf-98c32615ecd9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:04.337169 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c41df-1c94-4198-becf-98c32615ecd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:04.343822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-862c41df-1c94-4198-becf-98c32615ecd9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:04.343822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:04.343822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:04.343822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:04.343822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:04.343822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:04.343822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:04.343822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:04.343822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:04.343822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:04.343822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:04.343822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:04.343822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:04.343822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:04.343822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:04.343822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:04.344829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:04.344829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:04.344829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:04.344829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:04.344829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:04.344829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:04.344829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:04.344829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:04.344829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:04.344829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:04.344829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:04.344829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:04.344829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:04.344829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:04.344829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:04.344829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:04.345645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:04.345645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:04.345645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:04.345645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:04.345645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:04.345645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:04.345645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:04.345645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:04.345645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:04.345645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:04.345645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:04.345645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:08:04.348455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 114/215] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:04.355214 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c41df-1c94-4198-becf-98c32615ecd9 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:04.388368 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf203ee-6005-40ca-8d94-b0a25d28babe None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:04.419114 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:04.454100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a5c46-f57e-4cee-b655-5c6e57cc5b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:04.455342 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a5c46-f57e-4cee-b655-5c6e57cc5b50 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:04.455814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a5c46-f57e-4cee-b655-5c6e57cc5b50 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:04.456333 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a5c46-f57e-4cee-b655-5c6e57cc5b50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:04.458505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a5c46-f57e-4cee-b655-5c6e57cc5b50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:04.461197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a5c46-f57e-4cee-b655-5c6e57cc5b50 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:04.467068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 102/216] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:08:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:04.467068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a5c46-f57e-4cee-b655-5c6e57cc5b50 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:04.499854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c41df-1c94-4198-becf-98c32615ecd9 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:04.557277 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6e523-603e-4f38-96ca-6662185bf6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:04.558223 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6e523-603e-4f38-96ca-6662185bf6e2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:04.558689 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6e523-603e-4f38-96ca-6662185bf6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:04.559194 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6e523-603e-4f38-96ca-6662185bf6e2 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:04.560629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6e523-603e-4f38-96ca-6662185bf6e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:04.560959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6e523-603e-4f38-96ca-6662185bf6e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:04.563661 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6e523-603e-4f38-96ca-6662185bf6e2 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:04.585049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 115/217] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:04.585049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6e523-603e-4f38-96ca-6662185bf6e2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:04.625661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:04.634099 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a5c46-f57e-4cee-b655-5c6e57cc5b50 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:04.686893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101f938-c936-49b2-9cc5-c39a1cf51761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:04.687656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101f938-c936-49b2-9cc5-c39a1cf51761 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:04.687971 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101f938-c936-49b2-9cc5-c39a1cf51761 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:04.688247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101f938-c936-49b2-9cc5-c39a1cf51761 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:04.703217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6101f938-c936-49b2-9cc5-c39a1cf51761 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:04.709340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-6101f938-c936-49b2-9cc5-c39a1cf51761 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:08:04.709340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:04.709340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:04.709340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:04.709340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:04.709340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:04.709340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:04.709340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:04.709340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:04.709340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:04.709340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:04.709340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:04.709340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:04.709340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:04.709340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:04.709340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:04.710701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:04.710701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:04.710701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:04.710701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:04.710701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:04.710701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:04.710701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:04.710701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:04.710701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:04.710701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:04.710701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:04.710701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:04.710701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:04.710701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:04.710701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:04.710701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:04.712495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:04.712495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:04.712495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:04.712495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:04.712495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:04.712495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:04.712495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:04.712495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:04.712495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:04.712495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:04.712495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:08:04.712495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:04.712495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 103/218] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:08:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:04.712495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101f938-c936-49b2-9cc5-c39a1cf51761 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:04.738743 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6e523-603e-4f38-96ca-6662185bf6e2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:04.750432 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:04.788504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd4b16-1588-4ecf-981f-04e2e4114d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:04.789132 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd4b16-1588-4ecf-981f-04e2e4114d07 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:04.789670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd4b16-1588-4ecf-981f-04e2e4114d07 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:04.789670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd4b16-1588-4ecf-981f-04e2e4114d07 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:04.790344 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd4b16-1588-4ecf-981f-04e2e4114d07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:04.791848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd4b16-1588-4ecf-981f-04e2e4114d07 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:04.856075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101f938-c936-49b2-9cc5-c39a1cf51761 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:04.895689 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd464774-531b-4a32-80ae-c8bb63444ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:04.896334 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd464774-531b-4a32-80ae-c8bb63444ec2 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:04.896616 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd464774-531b-4a32-80ae-c8bb63444ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:04.896890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd464774-531b-4a32-80ae-c8bb63444ec2 None admin] PATH_INFO: `/v3/users/27e3bd2eea824cb98e3f10df2f33480c` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:04.905674 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 116/219] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:08:04 2022] POST /identity/v3/users => generated 297 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:08:04.906845 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322b42f-f3ff-4cde-ac81-e399b0483697 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:04.907218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322b42f-f3ff-4cde-ac81-e399b0483697 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:04.907513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322b42f-f3ff-4cde-ac81-e399b0483697 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:04.908773 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 117/220] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:04.910060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3322b42f-f3ff-4cde-ac81-e399b0483697 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:04.916455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd464774-531b-4a32-80ae-c8bb63444ec2 None admin] RBAC: Authorizing `identity:get_user(user_id=27e3bd2eea824cb98e3f10df2f33480c)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:04.918089 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd464774-531b-4a32-80ae-c8bb63444ec2 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:04.926254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 104/221] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:08:04 2022] GET /identity/v3/users/27e3bd2eea824cb98e3f10df2f33480c => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:04.928055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf8097-bd4d-4cd4-91a5-46fc33ea0ec0 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:04.928339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf8097-bd4d-4cd4-91a5-46fc33ea0ec0 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:04.928669 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf8097-bd4d-4cd4-91a5-46fc33ea0ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:04.949559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-6cdf8097-bd4d-4cd4-91a5-46fc33ea0ec0 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:04.952630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6cdf8097-bd4d-4cd4-91a5-46fc33ea0ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:04.970448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:05.072336 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3322b42f-f3ff-4cde-ac81-e399b0483697 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:05.087011 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 105/222] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:05.087011 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf8097-bd4d-4cd4-91a5-46fc33ea0ec0 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:05.161480 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c51351-03f2-4146-a94d-04b9a8f47291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:05.162498 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c51351-03f2-4146-a94d-04b9a8f47291 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:05.163096 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c51351-03f2-4146-a94d-04b9a8f47291 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:05.163602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c51351-03f2-4146-a94d-04b9a8f47291 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:05.164887 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c51351-03f2-4146-a94d-04b9a8f47291 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:05.165425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c51351-03f2-4146-a94d-04b9a8f47291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:05.169323 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c51351-03f2-4146-a94d-04b9a8f47291 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:05.188557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 118/223] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:05.197198 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c51351-03f2-4146-a94d-04b9a8f47291 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:05.261445 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:05.281545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf8097-bd4d-4cd4-91a5-46fc33ea0ec0 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:05.360255 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e0b0d-f66d-459c-b321-cdb833471953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:05.361896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e0b0d-f66d-459c-b321-cdb833471953 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:05.362957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e0b0d-f66d-459c-b321-cdb833471953 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:05.363796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e0b0d-f66d-459c-b321-cdb833471953 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:05.398883 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:05.412983 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c51351-03f2-4146-a94d-04b9a8f47291 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:05.502938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae8730-ece9-49a9-a31e-04e66d6e3445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:05.504160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae8730-ece9-49a9-a31e-04e66d6e3445 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:05.504644 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae8730-ece9-49a9-a31e-04e66d6e3445 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:05.505127 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae8730-ece9-49a9-a31e-04e66d6e3445 None admin] PATH_INFO: `/v3/users/2370ae01914048938a0317d4add8948b` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:05.508210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae8730-ece9-49a9-a31e-04e66d6e3445 None admin] RBAC: Authorizing `identity:get_user(user_id=2370ae01914048938a0317d4add8948b)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:05.511113 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae8730-ece9-49a9-a31e-04e66d6e3445 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:05.514278 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 119/224] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:08:05 2022] GET /identity/v3/users/2370ae01914048938a0317d4add8948b => generated 283 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:05.536939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae8730-ece9-49a9-a31e-04e66d6e3445 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:05.549831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e0b0d-f66d-459c-b321-cdb833471953 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:05.552931 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e0b0d-f66d-459c-b321-cdb833471953 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:05.742597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:05.756825 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae8730-ece9-49a9-a31e-04e66d6e3445 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:05.758137 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 106/225] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:08:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:08:05.763863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e0b0d-f66d-459c-b321-cdb833471953 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:05.830239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e6523-c6b1-4551-b179-6059595fd9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:05.832423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e6523-c6b1-4551-b179-6059595fd9cb None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:05.833424 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e6523-c6b1-4551-b179-6059595fd9cb None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:05.834108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e6523-c6b1-4551-b179-6059595fd9cb None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:05.836837 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e6523-c6b1-4551-b179-6059595fd9cb None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:05.840048 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e6523-c6b1-4551-b179-6059595fd9cb None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:05.844173 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 120/226] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:05 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:05.845912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e6523-c6b1-4551-b179-6059595fd9cb None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:05.879462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:05.968807 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e0b0d-f66d-459c-b321-cdb833471953 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:06.012983 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38165674-8aff-4e1a-b2ae-362f10a4aa8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:06.013545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38165674-8aff-4e1a-b2ae-362f10a4aa8e None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:06.013859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38165674-8aff-4e1a-b2ae-362f10a4aa8e None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:06.014140 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38165674-8aff-4e1a-b2ae-362f10a4aa8e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:06.029682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38165674-8aff-4e1a-b2ae-362f10a4aa8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:06.036018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-38165674-8aff-4e1a-b2ae-362f10a4aa8e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:08:06.036018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:06.036018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:06.036018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:06.036018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:06.036018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:06.036018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:06.036018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:06.036018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:06.036018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:06.036018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:06.036018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:06.036018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:06.036018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:06.036018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:06.036018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:06.037371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:06.037371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:06.037371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:06.037371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:06.037371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:06.037371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:06.037371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:06.037371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:06.037371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:06.037371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:06.037371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:06.037371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:06.037371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:06.037371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:06.037371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:06.037371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:06.038393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:06.038393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:06.038393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:06.038393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:06.038393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:06.038393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:06.038393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:06.038393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:06.038393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:06.038393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:06.038393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:08:06.038393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:06.038393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 107/227] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:08:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:06.038393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38165674-8aff-4e1a-b2ae-362f10a4aa8e None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:06.065606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e6523-c6b1-4551-b179-6059595fd9cb None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:06.127846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f01ca-6ec2-456c-85d5-6d5e0e97f7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:06.129126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f01ca-6ec2-456c-85d5-6d5e0e97f7dc None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:06.129701 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f01ca-6ec2-456c-85d5-6d5e0e97f7dc None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:06.130506 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f01ca-6ec2-456c-85d5-6d5e0e97f7dc None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:06.132596 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f01ca-6ec2-456c-85d5-6d5e0e97f7dc None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:06.134545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f01ca-6ec2-456c-85d5-6d5e0e97f7dc None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:06.137358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 121/228] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:05 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:06.138702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f01ca-6ec2-456c-85d5-6d5e0e97f7dc None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:06.202056 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:06.218758 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38165674-8aff-4e1a-b2ae-362f10a4aa8e None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:06.223592 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:06.293946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0a752-8b2a-4338-ad02-448a85b2904b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:06.296792 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0a752-8b2a-4338-ad02-448a85b2904b None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:06.297383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0a752-8b2a-4338-ad02-448a85b2904b None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:06.297880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0a752-8b2a-4338-ad02-448a85b2904b None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e/users/2370ae01914048938a0317d4add8948b/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:06.301825 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0a752-8b2a-4338-ad02-448a85b2904b None admin] RBAC: Authorizing `identity:create_grant(project_id=920dfa8ad35b4a429aea367ae3067b8e, user_id=2370ae01914048938a0317d4add8948b, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:06.304760 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0a752-8b2a-4338-ad02-448a85b2904b None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:06.364765 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f01ca-6ec2-456c-85d5-6d5e0e97f7dc None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:06.418734 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 108/229] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:08:06 2022] PUT /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e/users/2370ae01914048938a0317d4add8948b/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:06.426899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0a752-8b2a-4338-ad02-448a85b2904b None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:06.460531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc852d91-293e-49a4-945d-1a6d4b37b58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:06.462307 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc852d91-293e-49a4-945d-1a6d4b37b58e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:06.463347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc852d91-293e-49a4-945d-1a6d4b37b58e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:06.465135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc852d91-293e-49a4-945d-1a6d4b37b58e None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:06.467313 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc852d91-293e-49a4-945d-1a6d4b37b58e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:06.470827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc852d91-293e-49a4-945d-1a6d4b37b58e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:06.478469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc852d91-293e-49a4-945d-1a6d4b37b58e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:06.517145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 122/230] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:08:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:06.529376 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc852d91-293e-49a4-945d-1a6d4b37b58e None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:06.824839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0a752-8b2a-4338-ad02-448a85b2904b None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:06.907578 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:06.923195 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79921544-68c3-4504-871c-4b1d11b552a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:06.923849 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79921544-68c3-4504-871c-4b1d11b552a2 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:06.924158 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79921544-68c3-4504-871c-4b1d11b552a2 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:06.924436 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79921544-68c3-4504-871c-4b1d11b552a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:06.925821 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79921544-68c3-4504-871c-4b1d11b552a2 None admin] RBAC: Adding query filter params (role.id=bcc52f73cf03442e922a4757908334de, scope.project.id=920dfa8ad35b4a429aea367ae3067b8e, user.id=27e3bd2eea824cb98e3f10df2f33480c) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:06.926224 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79921544-68c3-4504-871c-4b1d11b552a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:06.928260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79921544-68c3-4504-871c-4b1d11b552a2 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:06.939979 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc852d91-293e-49a4-945d-1a6d4b37b58e None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:06.947016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 109/231] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:08:06 2022] GET /identity/v3/role_assignments?user.id=27e3bd2eea824cb98e3f10df2f33480c&scope.project.id=920dfa8ad35b4a429aea367ae3067b8e&role.id=bcc52f73cf03442e922a4757908334de => generated 259 bytes in 519 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:07.061773 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cf915-349b-4927-9f85-07966dc011b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:07.063281 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cf915-349b-4927-9f85-07966dc011b2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:07.063967 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cf915-349b-4927-9f85-07966dc011b2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:07.064699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cf915-349b-4927-9f85-07966dc011b2 None admin] PATH_INFO: `/v3/users/2370ae01914048938a0317d4add8948b` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:07.067869 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2cf915-349b-4927-9f85-07966dc011b2 None admin] RBAC: Authorizing `identity:get_user(user_id=2370ae01914048938a0317d4add8948b)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:07.071114 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2cf915-349b-4927-9f85-07966dc011b2 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:07.082774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 123/232] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:08:06 2022] GET /identity/v3/users/2370ae01914048938a0317d4add8948b => generated 283 bytes in 550 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:07.084633 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79921544-68c3-4504-871c-4b1d11b552a2 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:07.191659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:07.366310 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79921544-68c3-4504-871c-4b1d11b552a2 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:07.432028 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5de87-c289-4335-90f3-c0069cfa4dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:07.432884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5de87-c289-4335-90f3-c0069cfa4dbb None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:07.439163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5de87-c289-4335-90f3-c0069cfa4dbb None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:07.439163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5de87-c289-4335-90f3-c0069cfa4dbb None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:07.439163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5de87-c289-4335-90f3-c0069cfa4dbb None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:07.439163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5de87-c289-4335-90f3-c0069cfa4dbb None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:07.439163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 110/233] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:07 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:08:07.468195 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cf915-349b-4927-9f85-07966dc011b2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:07.760956 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cf915-349b-4927-9f85-07966dc011b2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:07.835891 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b66e83-4376-4ffa-b83a-bc0630413ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:07.837615 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b66e83-4376-4ffa-b83a-bc0630413ff8 None admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:07.838387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b66e83-4376-4ffa-b83a-bc0630413ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:07.839207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b66e83-4376-4ffa-b83a-bc0630413ff8 None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e/users/2370ae01914048938a0317d4add8948b/roles/fc0a7877dd0f4903a2a6fcc01a829c6a` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:07.843969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b66e83-4376-4ffa-b83a-bc0630413ff8 None admin] RBAC: Authorizing `identity:create_grant(project_id=920dfa8ad35b4a429aea367ae3067b8e, user_id=2370ae01914048938a0317d4add8948b, role_id=fc0a7877dd0f4903a2a6fcc01a829c6a)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:07.848059 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b66e83-4376-4ffa-b83a-bc0630413ff8 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:07.995984 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 124/234] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:08:07 2022] PUT /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e/users/2370ae01914048938a0317d4add8948b/roles/fc0a7877dd0f4903a2a6fcc01a829c6a => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:08.914535 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a22fb-2159-4ab6-ac90-1a4ddbdb10a4 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:08.915600 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a22fb-2159-4ab6-ac90-1a4ddbdb10a4 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:08.916140 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a22fb-2159-4ab6-ac90-1a4ddbdb10a4 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:08.918172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 111/235] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:08.924692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148c254-9795-4e0f-a693-3e1e7e9c5668 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:08.925833 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148c254-9795-4e0f-a693-3e1e7e9c5668 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:08.926165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148c254-9795-4e0f-a693-3e1e7e9c5668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:08.964456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b148c254-9795-4e0f-a693-3e1e7e9c5668 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:08.967409 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b148c254-9795-4e0f-a693-3e1e7e9c5668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:08.999895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:09.090414 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3422f-4266-4f10-97be-40a94ae4d67b None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:09.090846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3422f-4266-4f10-97be-40a94ae4d67b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:09.091043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3422f-4266-4f10-97be-40a94ae4d67b None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:09.092457 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 112/236] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:08:09.113337 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7659b-b6f8-4770-96f2-7d900305be08 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:09.113806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7659b-b6f8-4770-96f2-7d900305be08 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:09.114389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7659b-b6f8-4770-96f2-7d900305be08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:09.137305 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 125/237] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:09.139485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-90b7659b-b6f8-4770-96f2-7d900305be08 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:09.141102 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-90b7659b-b6f8-4770-96f2-7d900305be08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:09.253485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 113/238] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:09.685692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148c254-9795-4e0f-a693-3e1e7e9c5668 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:09.863832 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7659b-b6f8-4770-96f2-7d900305be08 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:09.892530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148c254-9795-4e0f-a693-3e1e7e9c5668 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:09.944099 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:10.008105 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b07f84-787a-4052-902a-ac0f67399297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:10.010205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b07f84-787a-4052-902a-ac0f67399297 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:10.011204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b07f84-787a-4052-902a-ac0f67399297 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:10.014491 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b07f84-787a-4052-902a-ac0f67399297 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:10.021078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b07f84-787a-4052-902a-ac0f67399297 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:10.024420 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b07f84-787a-4052-902a-ac0f67399297 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:10.028247 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 126/239] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:08:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:10.039244 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b07f84-787a-4052-902a-ac0f67399297 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:10.099377 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:10.216051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7659b-b6f8-4770-96f2-7d900305be08 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:10.329771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b07f84-787a-4052-902a-ac0f67399297 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:10.348282 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa254f-fb71-477f-9b44-17f98f530cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:10.349386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa254f-fb71-477f-9b44-17f98f530cd3 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:10.349914 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa254f-fb71-477f-9b44-17f98f530cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:10.350409 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa254f-fb71-477f-9b44-17f98f530cd3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:10.373172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38b092-ca1e-4f5d-93cf-701e8dc87cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:10.374054 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38b092-ca1e-4f5d-93cf-701e8dc87cd5 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:10.375318 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38b092-ca1e-4f5d-93cf-701e8dc87cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:10.375874 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38b092-ca1e-4f5d-93cf-701e8dc87cd5 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:10.379739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38b092-ca1e-4f5d-93cf-701e8dc87cd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:10.381460 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38b092-ca1e-4f5d-93cf-701e8dc87cd5 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:10.385208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa254f-fb71-477f-9b44-17f98f530cd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:10.390389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-24aa254f-fb71-477f-9b44-17f98f530cd3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:08:10.390389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:10.390389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:10.390389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:10.390389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:10.390389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:10.390389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:10.390389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:10.390389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:10.390389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:10.390389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:10.390389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:10.390389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:10.390389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:10.390389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:10.390389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:10.391586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:10.391586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:10.391586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:10.391586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:10.391586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:10.391586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:10.391586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:10.391586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:10.391586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:10.391586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:10.391586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:10.391586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:10.391586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:10.391586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:10.391586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:10.391586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:10.392476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:10.392476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:10.392476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:10.392476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:10.392476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:10.392476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:10.392476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:10.392476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:10.392476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:10.392476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:10.392476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:08:10.392476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:10.395144 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 114/240] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 532 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:10.396591 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faed73b-6501-4d2f-861f-ec5e4a8c1ddf None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:10.397020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faed73b-6501-4d2f-861f-ec5e4a8c1ddf None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:10.397447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faed73b-6501-4d2f-861f-ec5e4a8c1ddf None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:10.399315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 115/241] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:10.408207 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faed73b-6501-4d2f-861f-ec5e4a8c1ddf None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:10.499211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 127/242] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:08:10 2022] POST /identity/v3/users => generated 293 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:08:10.503258 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4366e-d54a-4c8c-ae71-3d29d43d59c7 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:10.504076 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4366e-d54a-4c8c-ae71-3d29d43d59c7 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:10.505986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4366e-d54a-4c8c-ae71-3d29d43d59c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:10.558358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-6ec4366e-d54a-4c8c-ae71-3d29d43d59c7 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:10.561090 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-6ec4366e-d54a-4c8c-ae71-3d29d43d59c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:10.641886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:10.670760 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faed73b-6501-4d2f-861f-ec5e4a8c1ddf None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:10.703351 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 128/243] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:10.711009 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc65d9-5490-4fc3-8b00-88cc83a48a79 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:10.711756 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc65d9-5490-4fc3-8b00-88cc83a48a79 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:10.712395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc65d9-5490-4fc3-8b00-88cc83a48a79 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:10.714470 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 129/244] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:10 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:10.720744 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b69eda-da99-475f-a80f-eb0bdf97db43 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:10.721304 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b69eda-da99-475f-a80f-eb0bdf97db43 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:10.721949 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b69eda-da99-475f-a80f-eb0bdf97db43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:10.732851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd472ba-fe0a-4a88-be3f-c9bc94fb717b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:10.733519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd472ba-fe0a-4a88-be3f-c9bc94fb717b None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:10.733804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd472ba-fe0a-4a88-be3f-c9bc94fb717b None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:10.734082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd472ba-fe0a-4a88-be3f-c9bc94fb717b None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:10.734891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd472ba-fe0a-4a88-be3f-c9bc94fb717b None admin] RBAC: Adding query filter params (name=reader) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:10.735171 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd472ba-fe0a-4a88-be3f-c9bc94fb717b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:10.739009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd472ba-fe0a-4a88-be3f-c9bc94fb717b None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:10.741577 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:10.751622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 116/245] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:10.758468 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd472ba-fe0a-4a88-be3f-c9bc94fb717b None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:10.768190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-27b69eda-da99-475f-a80f-eb0bdf97db43 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:10.769936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-27b69eda-da99-475f-a80f-eb0bdf97db43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:10.924581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 130/246] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:08:10.960380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd472ba-fe0a-4a88-be3f-c9bc94fb717b None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:11.007257 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ed647-7b64-4155-b752-79816a667461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:11.007860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ed647-7b64-4155-b752-79816a667461 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:11.008146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ed647-7b64-4155-b752-79816a667461 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:11.008426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ed647-7b64-4155-b752-79816a667461 None admin] PATH_INFO: `/v3/users/47fbab21dec64e58b748e24cc8031e4f` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:11.026539 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ed647-7b64-4155-b752-79816a667461 None admin] RBAC: Authorizing `identity:get_user(user_id=47fbab21dec64e58b748e24cc8031e4f)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:11.028284 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ed647-7b64-4155-b752-79816a667461 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:11.030343 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 117/247] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:08:10 2022] GET /identity/v3/users/47fbab21dec64e58b748e24cc8031e4f => generated 297 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:11.037316 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b69eda-da99-475f-a80f-eb0bdf97db43 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:11.088624 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ed647-7b64-4155-b752-79816a667461 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:11.119590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:11.239662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b69eda-da99-475f-a80f-eb0bdf97db43 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:11.310401 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09ddf7-c1fc-4795-a17f-b0bb24383ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:11.311844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09ddf7-c1fc-4795-a17f-b0bb24383ee7 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:11.311844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09ddf7-c1fc-4795-a17f-b0bb24383ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:11.311844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09ddf7-c1fc-4795-a17f-b0bb24383ee7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:11.315076 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:11.319231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ed647-7b64-4155-b752-79816a667461 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:11.359292 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f8a22-11ed-46a0-b2b3-555cc1ef6933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:11.359970 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f8a22-11ed-46a0-b2b3-555cc1ef6933 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:11.360255 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f8a22-11ed-46a0-b2b3-555cc1ef6933 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:11.360535 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f8a22-11ed-46a0-b2b3-555cc1ef6933 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:11.373914 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f8a22-11ed-46a0-b2b3-555cc1ef6933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:11.379774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-126f8a22-11ed-46a0-b2b3-555cc1ef6933 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:08:11.379774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:11.379774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:11.379774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:11.379774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:11.379774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:11.379774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:11.379774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:11.379774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:11.379774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:11.379774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:11.379774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:11.379774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:11.379774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:11.379774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:11.379774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:11.380727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:11.380727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:11.380727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:11.380727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:11.380727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:11.380727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:11.380727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:11.380727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:11.380727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:11.380727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:11.380727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:11.380727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:11.380727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:11.380727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:11.380727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:11.380727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:11.381509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:11.381509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:11.381509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:11.381509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:11.381509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:11.381509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:11.381509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:11.381509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:11.381509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:11.381509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:11.381509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:08:11.381509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:11.381509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 118/248] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:08:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:08:11.382248 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f8a22-11ed-46a0-b2b3-555cc1ef6933 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:11.445399 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09ddf7-c1fc-4795-a17f-b0bb24383ee7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:11.448283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09ddf7-c1fc-4795-a17f-b0bb24383ee7 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:11.484336 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:11.553728 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f8a22-11ed-46a0-b2b3-555cc1ef6933 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:11.651318 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7952d-5cd9-42e7-b429-c77bfe4f25b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:11.652376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7952d-5cd9-42e7-b429-c77bfe4f25b5 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:11.652879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7952d-5cd9-42e7-b429-c77bfe4f25b5 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:11.653280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7952d-5cd9-42e7-b429-c77bfe4f25b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:11.691649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7952d-5cd9-42e7-b429-c77bfe4f25b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:11.699720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 131/249] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:08:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 13:08:11.701303 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09ddf7-c1fc-4795-a17f-b0bb24383ee7 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:11.712259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-ddc7952d-5cd9-42e7-b429-c77bfe4f25b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:11.712259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:11.712259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:11.712259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:11.712259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:11.712259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:11.712259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:11.712259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:11.712259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:11.712259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:11.712259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:11.712259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:11.712259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:11.712259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:11.712259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:11.712259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:11.713372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:11.713372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:11.713372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:11.713372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:11.713372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:11.713372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:11.713372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:11.713372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:11.713372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:11.713372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:11.713372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:11.713372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:11.713372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:11.713372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:11.713372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:11.713372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:11.714293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:11.714293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:11.714293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:11.714293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:11.714293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:11.714293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:11.714293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:11.714293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:11.714293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:11.714293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:11.714293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:11.714293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:11.716800 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 119/250] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:11.719432 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d164f-80f6-42b8-8d05-e9bc1aa46d43 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:11.720057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d164f-80f6-42b8-8d05-e9bc1aa46d43 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:11.720536 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d164f-80f6-42b8-8d05-e9bc1aa46d43 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:11.723705 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 120/251] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:11.729036 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d164f-80f6-42b8-8d05-e9bc1aa46d43 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:11.788813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:11.904235 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d164f-80f6-42b8-8d05-e9bc1aa46d43 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:11.912643 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09ddf7-c1fc-4795-a17f-b0bb24383ee7 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:11.957002 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ba9fd-42f5-455d-a2dc-914e719c97aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:11.958024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ba9fd-42f5-455d-a2dc-914e719c97aa None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:11.958364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ba9fd-42f5-455d-a2dc-914e719c97aa None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:11.958670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ba9fd-42f5-455d-a2dc-914e719c97aa None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:11.959399 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ba9fd-42f5-455d-a2dc-914e719c97aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:11.959709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ba9fd-42f5-455d-a2dc-914e719c97aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:11.960670 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:11.962306 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ba9fd-42f5-455d-a2dc-914e719c97aa None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:11.979023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 132/252] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:11.979023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ba9fd-42f5-455d-a2dc-914e719c97aa None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:11.983128 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821da36c-1533-420e-8c18-f1b1929c9539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:11.984196 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821da36c-1533-420e-8c18-f1b1929c9539 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:11.984584 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821da36c-1533-420e-8c18-f1b1929c9539 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:11.985052 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821da36c-1533-420e-8c18-f1b1929c9539 None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:11.986633 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821da36c-1533-420e-8c18-f1b1929c9539 None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:11.989405 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821da36c-1533-420e-8c18-f1b1929c9539 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:11.993698 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 121/253] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:11 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:11.995313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb57af-af93-45da-bc58-4cb463cbccc9 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:11.995900 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb57af-af93-45da-bc58-4cb463cbccc9 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:11.996429 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb57af-af93-45da-bc58-4cb463cbccc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:12.020769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-bcbb57af-af93-45da-bc58-4cb463cbccc9 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:12.022168 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-bcbb57af-af93-45da-bc58-4cb463cbccc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:12.127208 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:12.135176 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ba9fd-42f5-455d-a2dc-914e719c97aa None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:12.152171 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 122/254] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:12.154162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb57af-af93-45da-bc58-4cb463cbccc9 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:12.178206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71762d1-9fcc-45f6-8bc5-fabfd1435a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:12.178788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71762d1-9fcc-45f6-8bc5-fabfd1435a4f None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:12.179106 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71762d1-9fcc-45f6-8bc5-fabfd1435a4f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:12.179794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71762d1-9fcc-45f6-8bc5-fabfd1435a4f None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:12.180066 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71762d1-9fcc-45f6-8bc5-fabfd1435a4f None admin] RBAC: Adding query filter params (name=member) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:12.180373 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71762d1-9fcc-45f6-8bc5-fabfd1435a4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:12.181764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71762d1-9fcc-45f6-8bc5-fabfd1435a4f None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:12.191832 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 133/255] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:12.197570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71762d1-9fcc-45f6-8bc5-fabfd1435a4f None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:12.297862 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb57af-af93-45da-bc58-4cb463cbccc9 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:12.340961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:12.355225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ab41b-f38a-43f1-bcbb-9e232c361229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:12.356449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ab41b-f38a-43f1-bcbb-9e232c361229 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:12.357008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ab41b-f38a-43f1-bcbb-9e232c361229 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:12.357490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ab41b-f38a-43f1-bcbb-9e232c361229 None admin] PATH_INFO: `/v3/users/2370ae01914048938a0317d4add8948b` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:12.359166 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ab41b-f38a-43f1-bcbb-9e232c361229 None admin] RBAC: Authorizing `identity:get_user(user_id=2370ae01914048938a0317d4add8948b)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:12.360991 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ab41b-f38a-43f1-bcbb-9e232c361229 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:12.371020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 123/256] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:08:12 2022] GET /identity/v3/users/2370ae01914048938a0317d4add8948b => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:12.373004 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ab41b-f38a-43f1-bcbb-9e232c361229 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:12.403682 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71762d1-9fcc-45f6-8bc5-fabfd1435a4f None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:12.472829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e59300-b23b-4bf1-8281-10cf765414df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:12.474026 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e59300-b23b-4bf1-8281-10cf765414df None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:12.474651 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e59300-b23b-4bf1-8281-10cf765414df None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:12.475186 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e59300-b23b-4bf1-8281-10cf765414df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:12.476611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e59300-b23b-4bf1-8281-10cf765414df None admin] RBAC: Adding query filter params (role.id=0503208661c44c4f8f52e30b4f21fff7, scope.project.id=920dfa8ad35b4a429aea367ae3067b8e, user.id=47fbab21dec64e58b748e24cc8031e4f) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:12.477172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e59300-b23b-4bf1-8281-10cf765414df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:12.479271 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e59300-b23b-4bf1-8281-10cf765414df None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:12.491551 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 134/257] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:08:12 2022] GET /identity/v3/role_assignments?user.id=47fbab21dec64e58b748e24cc8031e4f&scope.project.id=920dfa8ad35b4a429aea367ae3067b8e&role.id=0503208661c44c4f8f52e30b4f21fff7 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:12.492940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e59300-b23b-4bf1-8281-10cf765414df None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:12.573350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:12.667620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ab41b-f38a-43f1-bcbb-9e232c361229 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:12.743560 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f7f42-7777-4c63-8bd7-afee9fb84868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:12.744735 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f7f42-7777-4c63-8bd7-afee9fb84868 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:12.746738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f7f42-7777-4c63-8bd7-afee9fb84868 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:12.747305 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f7f42-7777-4c63-8bd7-afee9fb84868 None admin] PATH_INFO: `/v3/users/27e3bd2eea824cb98e3f10df2f33480c` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:12.749746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f7f42-7777-4c63-8bd7-afee9fb84868 None admin] RBAC: Authorizing `identity:get_user(user_id=27e3bd2eea824cb98e3f10df2f33480c)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:12.753121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f7f42-7777-4c63-8bd7-afee9fb84868 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:12.755532 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 124/258] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:08:12 2022] GET /identity/v3/users/27e3bd2eea824cb98e3f10df2f33480c => generated 297 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:12.758220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f7f42-7777-4c63-8bd7-afee9fb84868 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:12.766066 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e59300-b23b-4bf1-8281-10cf765414df None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:12.835250 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745bf9a-773f-4773-8836-ddda90f96972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:12.836828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745bf9a-773f-4773-8836-ddda90f96972 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:12.837647 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745bf9a-773f-4773-8836-ddda90f96972 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:12.838416 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745bf9a-773f-4773-8836-ddda90f96972 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:12.929774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:13.083223 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745bf9a-773f-4773-8836-ddda90f96972 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:13.084473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f7f42-7777-4c63-8bd7-afee9fb84868 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:13.084949 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745bf9a-773f-4773-8836-ddda90f96972 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:13.138833 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:13.149550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d14359-a027-427f-b910-a8663eecc496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:13.150316 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d14359-a027-427f-b910-a8663eecc496 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:13.150771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d14359-a027-427f-b910-a8663eecc496 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:13.151263 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d14359-a027-427f-b910-a8663eecc496 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:13.181020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d14359-a027-427f-b910-a8663eecc496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:13.199863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-e1d14359-a027-427f-b910-a8663eecc496 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:08:13.199863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:13.199863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:13.199863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:13.199863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:13.199863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:13.199863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:13.199863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:13.199863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:13.199863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:13.199863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:13.199863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:13.199863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:13.199863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:13.199863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:13.199863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:13.200823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:13.200823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:13.200823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:13.200823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:13.200823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:13.200823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:13.200823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:13.200823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:13.200823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:13.200823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:13.200823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:13.200823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:13.200823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:13.200823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:13.200823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:13.200823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:13.201618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:13.201618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:13.201618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:13.201618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:13.201618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:13.201618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:13.201618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:13.201618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:13.201618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:13.201618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:13.201618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:08:13.201618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:13.201618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 125/259] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:08:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 444 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:13.202851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d14359-a027-427f-b910-a8663eecc496 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:13.251133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 135/260] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:08:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 759 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:08:13.253316 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745bf9a-773f-4773-8836-ddda90f96972 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:13.423246 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745bf9a-773f-4773-8836-ddda90f96972 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:13.452019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:13.469887 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ca1f9-dccd-4c2d-a36b-79048312f2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:13.470855 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ca1f9-dccd-4c2d-a36b-79048312f2d8 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:13.471444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ca1f9-dccd-4c2d-a36b-79048312f2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:13.471875 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ca1f9-dccd-4c2d-a36b-79048312f2d8 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:13.472671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ca1f9-dccd-4c2d-a36b-79048312f2d8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:13.473014 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ca1f9-dccd-4c2d-a36b-79048312f2d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:13.474749 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ca1f9-dccd-4c2d-a36b-79048312f2d8 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:13.486613 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 136/261] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:08:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:13.488650 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d14359-a027-427f-b910-a8663eecc496 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:13.492370 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ca1f9-dccd-4c2d-a36b-79048312f2d8 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:13.542148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:13.584947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549edb14-cea9-41ef-bb10-a83a8b419473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:13.585840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549edb14-cea9-41ef-bb10-a83a8b419473 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:13.586132 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549edb14-cea9-41ef-bb10-a83a8b419473 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:13.586411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549edb14-cea9-41ef-bb10-a83a8b419473 None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:13.587826 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549edb14-cea9-41ef-bb10-a83a8b419473 None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:13.589708 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549edb14-cea9-41ef-bb10-a83a8b419473 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:13.598162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 126/262] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:13 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:13.598890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549edb14-cea9-41ef-bb10-a83a8b419473 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:13.633155 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ca1f9-dccd-4c2d-a36b-79048312f2d8 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:13.681791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3afca-1d92-4a5b-adaa-e22a0b948643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:13.682718 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3afca-1d92-4a5b-adaa-e22a0b948643 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:13.683236 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3afca-1d92-4a5b-adaa-e22a0b948643 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:13.683685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3afca-1d92-4a5b-adaa-e22a0b948643 None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:13.685476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3afca-1d92-4a5b-adaa-e22a0b948643 None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:13.687368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3afca-1d92-4a5b-adaa-e22a0b948643 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:13.690402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 137/263] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:13 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:13.691580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3afca-1d92-4a5b-adaa-e22a0b948643 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:13.789769 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:13.916876 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549edb14-cea9-41ef-bb10-a83a8b419473 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:13.964863 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3afca-1d92-4a5b-adaa-e22a0b948643 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:14.044120 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653640a-5003-4d2a-abee-915a9bea3cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:14.045234 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653640a-5003-4d2a-abee-915a9bea3cc2 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:14.045728 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653640a-5003-4d2a-abee-915a9bea3cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:14.046265 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653640a-5003-4d2a-abee-915a9bea3cc2 None admin] PATH_INFO: `/v3/users/2370ae01914048938a0317d4add8948b` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:14.050202 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653640a-5003-4d2a-abee-915a9bea3cc2 None admin] RBAC: Authorizing `identity:get_user(user_id=2370ae01914048938a0317d4add8948b)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:14.052132 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653640a-5003-4d2a-abee-915a9bea3cc2 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:14.055525 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 127/264] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:08:13 2022] GET /identity/v3/users/2370ae01914048938a0317d4add8948b => generated 283 bytes in 457 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:14.056762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653640a-5003-4d2a-abee-915a9bea3cc2 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:14.095682 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e673e3-5f2f-4ece-9a92-59073e41bee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:14.098400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e673e3-5f2f-4ece-9a92-59073e41bee7 None admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:14.099274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e673e3-5f2f-4ece-9a92-59073e41bee7 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:14.100942 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e673e3-5f2f-4ece-9a92-59073e41bee7 None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e/users/27e3bd2eea824cb98e3f10df2f33480c/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:14.116079 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e673e3-5f2f-4ece-9a92-59073e41bee7 None admin] RBAC: Authorizing `identity:create_grant(project_id=920dfa8ad35b4a429aea367ae3067b8e, user_id=27e3bd2eea824cb98e3f10df2f33480c, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:14.117840 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e673e3-5f2f-4ece-9a92-59073e41bee7 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:14.207033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 138/265] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:08:13 2022] PUT /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e/users/27e3bd2eea824cb98e3f10df2f33480c/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:14.209617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e673e3-5f2f-4ece-9a92-59073e41bee7 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:14.215035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:14.427086 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653640a-5003-4d2a-abee-915a9bea3cc2 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:14.550878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e673e3-5f2f-4ece-9a92-59073e41bee7 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:14.561947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e985ab-9f61-46cb-843f-838642d71585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:14.563019 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e985ab-9f61-46cb-843f-838642d71585 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:14.563531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e985ab-9f61-46cb-843f-838642d71585 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:14.564007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e985ab-9f61-46cb-843f-838642d71585 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:14.566630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e985ab-9f61-46cb-843f-838642d71585 None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=920dfa8ad35b4a429aea367ae3067b8e, user.id=2370ae01914048938a0317d4add8948b) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:14.567228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e985ab-9f61-46cb-843f-838642d71585 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:14.569996 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e985ab-9f61-46cb-843f-838642d71585 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:14.602163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 128/266] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:08:14 2022] GET /identity/v3/role_assignments?user.id=2370ae01914048938a0317d4add8948b&scope.project.id=920dfa8ad35b4a429aea367ae3067b8e&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 608 bytes in 546 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:14.603759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:14.606624 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de8864-b7bf-4987-a320-d72dbe8fdd26 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:14.607540 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de8864-b7bf-4987-a320-d72dbe8fdd26 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:14.608261 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de8864-b7bf-4987-a320-d72dbe8fdd26 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:14.611637 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 129/267] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:14.622015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b62b0f-b3d7-43af-bb3c-98c8fb44236a None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:14.626107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b62b0f-b3d7-43af-bb3c-98c8fb44236a None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:14.626107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b62b0f-b3d7-43af-bb3c-98c8fb44236a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:14.681295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-82b62b0f-b3d7-43af-bb3c-98c8fb44236a None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:14.683403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-82b62b0f-b3d7-43af-bb3c-98c8fb44236a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:14.683668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1f181-a3c1-4099-9aef-5a5662791073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:14.684976 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1f181-a3c1-4099-9aef-5a5662791073 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:14.685688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1f181-a3c1-4099-9aef-5a5662791073 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:14.686368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1f181-a3c1-4099-9aef-5a5662791073 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:14.866273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1f181-a3c1-4099-9aef-5a5662791073 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:14.869171 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1f181-a3c1-4099-9aef-5a5662791073 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:14.873825 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 130/268] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:08:14.876510 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ed085-79be-4b7f-b55d-310ac90361df None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:14.877224 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ed085-79be-4b7f-b55d-310ac90361df None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:14.877872 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ed085-79be-4b7f-b55d-310ac90361df None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:14.881147 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 131/269] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:14.888286 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01d260-7526-4070-a594-ba5531518083 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:14.889118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01d260-7526-4070-a594-ba5531518083 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:14.889801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01d260-7526-4070-a594-ba5531518083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:14.956930 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-cc01d260-7526-4070-a594-ba5531518083 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:14.960171 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-cc01d260-7526-4070-a594-ba5531518083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:15.088716 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:15.131744 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 132/270] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:15.175826 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 139/271] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:08:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 968 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:08:15.184682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01d260-7526-4070-a594-ba5531518083 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:15.389375 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01d260-7526-4070-a594-ba5531518083 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:15.455751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08504130-d4b8-4ab8-9fca-415d305b69b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:15.457386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08504130-d4b8-4ab8-9fca-415d305b69b3 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:15.458158 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08504130-d4b8-4ab8-9fca-415d305b69b3 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:15.458652 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08504130-d4b8-4ab8-9fca-415d305b69b3 None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:15.460830 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08504130-d4b8-4ab8-9fca-415d305b69b3 None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:15.463697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08504130-d4b8-4ab8-9fca-415d305b69b3 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:15.470478 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 133/272] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:15 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:15.479486 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1f181-a3c1-4099-9aef-5a5662791073 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:15.512151 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:15.554885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08504130-d4b8-4ab8-9fca-415d305b69b3 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:15.667592 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:15.772994 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08504130-d4b8-4ab8-9fca-415d305b69b3 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:15.848955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b45c4f-6433-4202-85c6-d8620694fb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:15.850455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b45c4f-6433-4202-85c6-d8620694fb11 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:15.850895 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b45c4f-6433-4202-85c6-d8620694fb11 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:15.851364 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b45c4f-6433-4202-85c6-d8620694fb11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:15.864038 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1f181-a3c1-4099-9aef-5a5662791073 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:15.873811 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b45c4f-6433-4202-85c6-d8620694fb11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:15.885570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-90b45c4f-6433-4202-85c6-d8620694fb11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:15.885570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:15.885570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:15.885570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:15.885570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:15.885570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:15.885570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:15.885570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:15.885570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:15.885570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:15.885570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:15.885570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:15.885570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:15.885570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:15.885570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:15.885570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:15.887277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:15.887277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:15.887277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:15.887277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:15.887277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:15.887277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:15.887277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:15.887277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:15.887277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:15.887277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:15.887277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:15.887277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:15.887277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:15.887277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:15.887277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:15.887277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:15.888871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:15.888871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:15.888871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:15.888871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:15.888871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:15.888871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:15.888871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:15.888871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:15.888871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:15.888871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:15.888871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:15.888871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:15.888871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 134/273] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:15.906082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b45c4f-6433-4202-85c6-d8620694fb11 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:15.956045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f172a-a61a-4e42-a047-6127ffd4cac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:15.956911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f172a-a61a-4e42-a047-6127ffd4cac0 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:15.957394 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f172a-a61a-4e42-a047-6127ffd4cac0 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:15.958106 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f172a-a61a-4e42-a047-6127ffd4cac0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:15.959611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f172a-a61a-4e42-a047-6127ffd4cac0 None admin] RBAC: Adding query filter params (role.id=fc0a7877dd0f4903a2a6fcc01a829c6a, scope.project.id=920dfa8ad35b4a429aea367ae3067b8e, user.id=2370ae01914048938a0317d4add8948b) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:15.960156 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f172a-a61a-4e42-a047-6127ffd4cac0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:15.963285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f172a-a61a-4e42-a047-6127ffd4cac0 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:15.979523 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 140/274] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:08:15 2022] GET /identity/v3/role_assignments?user.id=2370ae01914048938a0317d4add8948b&scope.project.id=920dfa8ad35b4a429aea367ae3067b8e&role.id=fc0a7877dd0f4903a2a6fcc01a829c6a => generated 608 bytes in 502 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:08:15.981177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f172a-a61a-4e42-a047-6127ffd4cac0 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:16.122763 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b45c4f-6433-4202-85c6-d8620694fb11 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:16.169256 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:16.200328 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b9a1a-511f-495c-8e0f-e2d5af5965fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:16.201999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b9a1a-511f-495c-8e0f-e2d5af5965fd None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:16.202807 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b9a1a-511f-495c-8e0f-e2d5af5965fd None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:16.203682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b9a1a-511f-495c-8e0f-e2d5af5965fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:16.212178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748b9a1a-511f-495c-8e0f-e2d5af5965fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:16.212178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748b9a1a-511f-495c-8e0f-e2d5af5965fd None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:16.214608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 135/275] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:08:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:16.225051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b9a1a-511f-495c-8e0f-e2d5af5965fd None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:16.243584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:16.287030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f172a-a61a-4e42-a047-6127ffd4cac0 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:16.360610 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e94bc3-a171-4c25-a0ea-d3d9c4e9d9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:16.362630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e94bc3-a171-4c25-a0ea-d3d9c4e9d9a4 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:16.363433 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e94bc3-a171-4c25-a0ea-d3d9c4e9d9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:16.365197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e94bc3-a171-4c25-a0ea-d3d9c4e9d9a4 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:16.367351 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e94bc3-a171-4c25-a0ea-d3d9c4e9d9a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:16.368512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e94bc3-a171-4c25-a0ea-d3d9c4e9d9a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:16.369947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e94bc3-a171-4c25-a0ea-d3d9c4e9d9a4 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:16.386127 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 141/276] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:16.396099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e94bc3-a171-4c25-a0ea-d3d9c4e9d9a4 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:16.448407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b9a1a-511f-495c-8e0f-e2d5af5965fd None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:16.515422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13154c2b-04cd-4649-9e79-8acd9a3dc3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:16.516593 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13154c2b-04cd-4649-9e79-8acd9a3dc3c5 None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:16.517531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13154c2b-04cd-4649-9e79-8acd9a3dc3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:16.518037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13154c2b-04cd-4649-9e79-8acd9a3dc3c5 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:16.520239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13154c2b-04cd-4649-9e79-8acd9a3dc3c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:16.522611 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13154c2b-04cd-4649-9e79-8acd9a3dc3c5 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:16.615752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e94bc3-a171-4c25-a0ea-d3d9c4e9d9a4 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:16.659956 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 136/277] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:08:16 2022] POST /identity/v3/users => generated 293 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:08:16.669901 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50f9d0-a0f6-4ebb-9e89-ffe76eec7212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:16.670664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50f9d0-a0f6-4ebb-9e89-ffe76eec7212 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:16.670955 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50f9d0-a0f6-4ebb-9e89-ffe76eec7212 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:16.671263 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50f9d0-a0f6-4ebb-9e89-ffe76eec7212 None admin] PATH_INFO: `/v3/users/3dfbc9f1a02f4720aed732eb352dbb5e` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:16.687556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:16.700323 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50f9d0-a0f6-4ebb-9e89-ffe76eec7212 None admin] RBAC: Authorizing `identity:get_user(user_id=3dfbc9f1a02f4720aed732eb352dbb5e)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:16.703192 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50f9d0-a0f6-4ebb-9e89-ffe76eec7212 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:16.706717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 142/278] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:08:16 2022] GET /identity/v3/users/3dfbc9f1a02f4720aed732eb352dbb5e => generated 293 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:08:16.716338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13154c2b-04cd-4649-9e79-8acd9a3dc3c5 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:16.777353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:16.887578 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13154c2b-04cd-4649-9e79-8acd9a3dc3c5 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:16.950005 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29086d30-a1e9-438b-8045-d6d3d58edd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:16.951759 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29086d30-a1e9-438b-8045-d6d3d58edd46 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:16.952552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29086d30-a1e9-438b-8045-d6d3d58edd46 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:16.953183 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29086d30-a1e9-438b-8045-d6d3d58edd46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:16.954461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29086d30-a1e9-438b-8045-d6d3d58edd46 None admin] RBAC: Adding query filter params (role.id=bcc52f73cf03442e922a4757908334de, user.id=3dfbc9f1a02f4720aed732eb352dbb5e, scope.system=all) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:16.955151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29086d30-a1e9-438b-8045-d6d3d58edd46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:16.958853 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29086d30-a1e9-438b-8045-d6d3d58edd46 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:16.970893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 137/279] 10.222.0.62 () {56 vars in 1308 bytes} [Wed Sep 7 13:08:16 2022] GET /identity/v3/role_assignments?user.id=3dfbc9f1a02f4720aed732eb352dbb5e&scope.system=all&role.id=bcc52f73cf03442e922a4757908334de => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:17.130485 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1428cb-cd5f-4c40-93e4-154bd34d024b None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:17.131038 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1428cb-cd5f-4c40-93e4-154bd34d024b None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:17.131409 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1428cb-cd5f-4c40-93e4-154bd34d024b None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:17.134138 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 143/280] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:17.144223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80855e0-fa69-4924-8e17-0ed7bfec4ef4 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:17.144705 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80855e0-fa69-4924-8e17-0ed7bfec4ef4 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:17.145164 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80855e0-fa69-4924-8e17-0ed7bfec4ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:17.181302 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-f80855e0-fa69-4924-8e17-0ed7bfec4ef4 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:17.183312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f80855e0-fa69-4924-8e17-0ed7bfec4ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:17.305180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 138/281] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:17.683420 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1428cb-cd5f-4c40-93e4-154bd34d024b None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:17.915791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1428cb-cd5f-4c40-93e4-154bd34d024b None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:17.985784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632de477-1ffd-4a88-8f2c-dd9b91252294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:17.986712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632de477-1ffd-4a88-8f2c-dd9b91252294 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:17.987843 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632de477-1ffd-4a88-8f2c-dd9b91252294 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:17.988317 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632de477-1ffd-4a88-8f2c-dd9b91252294 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:18.009607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632de477-1ffd-4a88-8f2c-dd9b91252294 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:18.020592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-632de477-1ffd-4a88-8f2c-dd9b91252294 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:08:18.020592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:18.020592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:18.020592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:18.020592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:18.020592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:18.020592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:18.020592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:18.020592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:18.020592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:18.020592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:18.020592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:18.020592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:18.020592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:18.020592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:18.020592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:18.021605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:18.021605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:18.021605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:18.021605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:18.021605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:18.021605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:18.021605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:18.021605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:18.021605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:18.021605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:18.021605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:18.021605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:18.021605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:18.021605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:18.021605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:18.021605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:18.022662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:18.022662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:18.022662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:18.022662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:18.022662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:18.022662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:18.022662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:18.022662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:18.022662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:18.022662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:18.022662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:08:18.022662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:08:18.023337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 144/282] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:18.032651 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80855e0-fa69-4924-8e17-0ed7bfec4ef4 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:18.102838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:18.250761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80855e0-fa69-4924-8e17-0ed7bfec4ef4 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:18.324857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542de1d-a29d-4d00-888f-1fc89033c42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:18.326034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542de1d-a29d-4d00-888f-1fc89033c42b None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:18.326542 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542de1d-a29d-4d00-888f-1fc89033c42b None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:18.327063 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542de1d-a29d-4d00-888f-1fc89033c42b None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:18.328443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542de1d-a29d-4d00-888f-1fc89033c42b None admin] RBAC: Adding query filter params (name=reader) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:18.328992 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542de1d-a29d-4d00-888f-1fc89033c42b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:18.332350 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542de1d-a29d-4d00-888f-1fc89033c42b None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:18.350774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 139/283] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:18.360121 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632de477-1ffd-4a88-8f2c-dd9b91252294 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:18.510670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:18.604160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632de477-1ffd-4a88-8f2c-dd9b91252294 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:18.676819 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed461d6-db08-41aa-983c-904646dd7105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:18.678084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed461d6-db08-41aa-983c-904646dd7105 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:18.678585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed461d6-db08-41aa-983c-904646dd7105 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:18.679075 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed461d6-db08-41aa-983c-904646dd7105 None admin] PATH_INFO: `/v3/users/47fbab21dec64e58b748e24cc8031e4f` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:18.681311 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed461d6-db08-41aa-983c-904646dd7105 None admin] RBAC: Authorizing `identity:get_user(user_id=47fbab21dec64e58b748e24cc8031e4f)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:18.684125 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed461d6-db08-41aa-983c-904646dd7105 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:18.689050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 145/284] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:08:18 2022] GET /identity/v3/users/47fbab21dec64e58b748e24cc8031e4f => generated 297 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:18.704470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542de1d-a29d-4d00-888f-1fc89033c42b None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:18.945233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:18.978098 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542de1d-a29d-4d00-888f-1fc89033c42b None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:19.028179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36ce6d-ebe7-420f-9e92-63a0e65fcf82 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:19.028905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36ce6d-ebe7-420f-9e92-63a0e65fcf82 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:19.029505 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36ce6d-ebe7-420f-9e92-63a0e65fcf82 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:19.032341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 146/285] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:08:19.042650 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d4a58-ef57-4d93-9ddc-feed319e5fb3 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:19.043425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d4a58-ef57-4d93-9ddc-feed319e5fb3 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:19.043967 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d4a58-ef57-4d93-9ddc-feed319e5fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:19.048189 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91a4e6-8378-49da-81ce-43a862345638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:19.050490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91a4e6-8378-49da-81ce-43a862345638 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:19.050962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91a4e6-8378-49da-81ce-43a862345638 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:19.051405 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91a4e6-8378-49da-81ce-43a862345638 None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:19.052845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91a4e6-8378-49da-81ce-43a862345638 None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:19.054484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91a4e6-8378-49da-81ce-43a862345638 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:19.056611 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 140/286] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:18 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:19.066403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91a4e6-8378-49da-81ce-43a862345638 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:19.080191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-7b7d4a58-ef57-4d93-9ddc-feed319e5fb3 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:19.083366 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7b7d4a58-ef57-4d93-9ddc-feed319e5fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:19.302419 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 147/287] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:08:19.376345 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91a4e6-8378-49da-81ce-43a862345638 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:19.435336 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34037c-c1ff-466b-a7ba-c1985a1bb822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:19.436539 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34037c-c1ff-466b-a7ba-c1985a1bb822 None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:19.437012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34037c-c1ff-466b-a7ba-c1985a1bb822 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:19.437487 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34037c-c1ff-466b-a7ba-c1985a1bb822 None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e/users/47fbab21dec64e58b748e24cc8031e4f/roles/0503208661c44c4f8f52e30b4f21fff7` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:19.441101 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34037c-c1ff-466b-a7ba-c1985a1bb822 None admin] RBAC: Authorizing `identity:create_grant(project_id=920dfa8ad35b4a429aea367ae3067b8e, user_id=47fbab21dec64e58b748e24cc8031e4f, role_id=0503208661c44c4f8f52e30b4f21fff7)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:19.444525 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34037c-c1ff-466b-a7ba-c1985a1bb822 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:19.525968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 141/288] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:08:19 2022] PUT /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e/users/47fbab21dec64e58b748e24cc8031e4f/roles/0503208661c44c4f8f52e30b4f21fff7 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:08:19.839514 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d4a58-ef57-4d93-9ddc-feed319e5fb3 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:19.954465 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:20.026099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d4a58-ef57-4d93-9ddc-feed319e5fb3 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:20.063971 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17419a68-6c0e-4177-b8de-5124b17b59f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:20.064623 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17419a68-6c0e-4177-b8de-5124b17b59f2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:20.065613 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17419a68-6c0e-4177-b8de-5124b17b59f2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:20.066692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17419a68-6c0e-4177-b8de-5124b17b59f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:20.078034 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17419a68-6c0e-4177-b8de-5124b17b59f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:20.084224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-17419a68-6c0e-4177-b8de-5124b17b59f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:20.084224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:20.084224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:20.084224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:20.084224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:20.084224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:20.084224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:20.084224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:20.084224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:20.084224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:20.084224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:20.084224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:20.084224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:20.084224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:20.084224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:20.084224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:20.085219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:20.085219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:20.085219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:20.085219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:20.085219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:20.085219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:20.085219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:20.085219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:20.085219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:20.085219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:20.085219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:20.085219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:20.085219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:20.085219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:20.085219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:20.085219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:20.086058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:20.086058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:20.086058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:20.086058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:20.086058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:20.086058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:20.086058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:20.086058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:20.086058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:20.086058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:20.086058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:20.086058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:08:20.086058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 148/289] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:20.092202 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34037c-c1ff-466b-a7ba-c1985a1bb822 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:20.128960 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:20.233078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34037c-c1ff-466b-a7ba-c1985a1bb822 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:20.276301 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a266f-b100-40cd-92cf-ca839f4423b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:20.277012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a266f-b100-40cd-92cf-ca839f4423b7 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:20.277303 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a266f-b100-40cd-92cf-ca839f4423b7 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:20.277714 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a266f-b100-40cd-92cf-ca839f4423b7 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:20.278430 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a266f-b100-40cd-92cf-ca839f4423b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:20.278734 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a266f-b100-40cd-92cf-ca839f4423b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:20.280227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a266f-b100-40cd-92cf-ca839f4423b7 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:20.292514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 142/290] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:20.299095 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17419a68-6c0e-4177-b8de-5124b17b59f2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:20.450110 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17419a68-6c0e-4177-b8de-5124b17b59f2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:20.473217 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:20.503405 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c06ea6-354e-4c67-8470-ea219e8a22c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:20.504056 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c06ea6-354e-4c67-8470-ea219e8a22c8 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:20.504340 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c06ea6-354e-4c67-8470-ea219e8a22c8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:20.504613 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c06ea6-354e-4c67-8470-ea219e8a22c8 None admin] PATH_INFO: `/v3/users/27e3bd2eea824cb98e3f10df2f33480c` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:20.507954 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c06ea6-354e-4c67-8470-ea219e8a22c8 None admin] RBAC: Authorizing `identity:get_user(user_id=27e3bd2eea824cb98e3f10df2f33480c)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:20.509832 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c06ea6-354e-4c67-8470-ea219e8a22c8 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:20.512231 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 149/291] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:08:20 2022] GET /identity/v3/users/27e3bd2eea824cb98e3f10df2f33480c => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:20.519054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a266f-b100-40cd-92cf-ca839f4423b7 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:20.647342 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:20.686175 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a266f-b100-40cd-92cf-ca839f4423b7 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:20.741506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960364a2-5847-4674-ae82-45a17f74cc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:20.741506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960364a2-5847-4674-ae82-45a17f74cc9f None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:20.744482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960364a2-5847-4674-ae82-45a17f74cc9f None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:20.744482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960364a2-5847-4674-ae82-45a17f74cc9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:20.901160 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960364a2-5847-4674-ae82-45a17f74cc9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:20.904201 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960364a2-5847-4674-ae82-45a17f74cc9f None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:20.949899 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8933945-7be4-4425-bcfe-59ecf926866f None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:20.950768 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8933945-7be4-4425-bcfe-59ecf926866f None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:20.950768 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8933945-7be4-4425-bcfe-59ecf926866f None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:20.952163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 150/292] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:08:20.961166 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc60036-a08c-4644-86a7-7791b4d09618 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:20.961557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc60036-a08c-4644-86a7-7791b4d09618 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:20.961832 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc60036-a08c-4644-86a7-7791b4d09618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:20.992814 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-5bc60036-a08c-4644-86a7-7791b4d09618 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:20.994144 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-5bc60036-a08c-4644-86a7-7791b4d09618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:21.093921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:21.137240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 143/293] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:08:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:08:21.138905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20d543-0ed6-4535-8a88-e6ee31d1f9b3 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:21.139461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20d543-0ed6-4535-8a88-e6ee31d1f9b3 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:21.139962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20d543-0ed6-4535-8a88-e6ee31d1f9b3 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:21.140439 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 151/294] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:08:21.141875 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 144/295] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:21.144269 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e495e41-1b42-4c84-82c8-66e7c4be4403 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:21.145123 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e495e41-1b42-4c84-82c8-66e7c4be4403 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:21.145888 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e495e41-1b42-4c84-82c8-66e7c4be4403 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:21.151078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de20d543-0ed6-4535-8a88-e6ee31d1f9b3 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:21.153518 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 152/296] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:21 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:21.158662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65193d28-51e7-4d08-92cd-e0658b891980 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:21.160042 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65193d28-51e7-4d08-92cd-e0658b891980 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:21.160912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65193d28-51e7-4d08-92cd-e0658b891980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:21.193482 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-65193d28-51e7-4d08-92cd-e0658b891980 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:21.196027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-65193d28-51e7-4d08-92cd-e0658b891980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:21.344103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de20d543-0ed6-4535-8a88-e6ee31d1f9b3 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:21.356457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 153/297] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:21.358289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f94f9-20fa-4136-b5a3-ba2795502a87 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:21.358577 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f94f9-20fa-4136-b5a3-ba2795502a87 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:21.358937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f94f9-20fa-4136-b5a3-ba2795502a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:21.409550 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-507f94f9-20fa-4136-b5a3-ba2795502a87 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:21.418820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-507f94f9-20fa-4136-b5a3-ba2795502a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:21.474503 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2290b-c586-4eda-8449-a5d5d00c04db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:21.483977 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2290b-c586-4eda-8449-a5d5d00c04db None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:21.484314 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2290b-c586-4eda-8449-a5d5d00c04db None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:21.484563 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2290b-c586-4eda-8449-a5d5d00c04db None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:21.491954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba2290b-c586-4eda-8449-a5d5d00c04db None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:21.493524 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba2290b-c586-4eda-8449-a5d5d00c04db None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:21.495376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 145/298] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:21 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:21.501549 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2290b-c586-4eda-8449-a5d5d00c04db None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:21.537949 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:21.605964 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:21.623225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 154/299] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:21.624408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f94f9-20fa-4136-b5a3-ba2795502a87 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:21.701550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2290b-c586-4eda-8449-a5d5d00c04db None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:21.746395 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70a086-1957-4a08-8d43-3d52f4f22312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:21.747079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70a086-1957-4a08-8d43-3d52f4f22312 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:21.747358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70a086-1957-4a08-8d43-3d52f4f22312 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:21.747630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70a086-1957-4a08-8d43-3d52f4f22312 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:21.748391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e70a086-1957-4a08-8d43-3d52f4f22312 None admin] RBAC: Adding query filter params (role.id=bcc52f73cf03442e922a4757908334de, scope.project.id=920dfa8ad35b4a429aea367ae3067b8e, user.id=27e3bd2eea824cb98e3f10df2f33480c) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:21.748702 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e70a086-1957-4a08-8d43-3d52f4f22312 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:21.750429 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e70a086-1957-4a08-8d43-3d52f4f22312 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:21.767617 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 146/300] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:08:21 2022] GET /identity/v3/role_assignments?user.id=27e3bd2eea824cb98e3f10df2f33480c&scope.project.id=920dfa8ad35b4a429aea367ae3067b8e&role.id=bcc52f73cf03442e922a4757908334de => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:08:21.769330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70a086-1957-4a08-8d43-3d52f4f22312 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:21.830284 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f94f9-20fa-4136-b5a3-ba2795502a87 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:21.933828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff79db7-76ee-4aa5-8225-aa3e66bf512f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:21.934853 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff79db7-76ee-4aa5-8225-aa3e66bf512f None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:21.935318 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff79db7-76ee-4aa5-8225-aa3e66bf512f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:21.935600 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff79db7-76ee-4aa5-8225-aa3e66bf512f None admin] PATH_INFO: `/v3/roles/member` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:21.947489 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70a086-1957-4a08-8d43-3d52f4f22312 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:21.962272 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff79db7-76ee-4aa5-8225-aa3e66bf512f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:21.970787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-8ff79db7-76ee-4aa5-8225-aa3e66bf512f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:21.970787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:21.970787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:21.970787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:21.970787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:21.970787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:21.970787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:21.970787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:21.970787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:21.970787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:21.970787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:21.970787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:21.970787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:21.970787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:21.970787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:21.970787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:21.973355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:21.973355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:21.973355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:21.973355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:21.973355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:21.973355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:21.973355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:21.973355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:21.973355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:21.973355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:21.973355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:21.973355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:21.973355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:21.973355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:21.973355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:21.973355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:21.975489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:21.975489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:21.975489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:21.975489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:21.975489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:21.975489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:21.975489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:21.975489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:21.975489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:21.975489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:21.975489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:21.975489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:08:21.977324 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:21.977506 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 155/301] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:21.977506 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff79db7-76ee-4aa5-8225-aa3e66bf512f None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:22.029805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7a036-c3ec-4859-962e-fd83c81a510c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:22.031159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7a036-c3ec-4859-962e-fd83c81a510c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:22.031802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7a036-c3ec-4859-962e-fd83c81a510c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:22.032382 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7a036-c3ec-4859-962e-fd83c81a510c None admin] PATH_INFO: `/v3/domains/default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:22.035400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:22.037399 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7a036-c3ec-4859-962e-fd83c81a510c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:22.040391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7a036-c3ec-4859-962e-fd83c81a510c None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:22.044326 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 147/302] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:08:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:22.053004 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7a036-c3ec-4859-962e-fd83c81a510c None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:22.164139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff79db7-76ee-4aa5-8225-aa3e66bf512f None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:22.237159 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c6b84-e7b8-4fb6-ab9c-310feca61323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:22.239037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c6b84-e7b8-4fb6-ab9c-310feca61323 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:22.239836 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c6b84-e7b8-4fb6-ab9c-310feca61323 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:22.240571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c6b84-e7b8-4fb6-ab9c-310feca61323 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:22.264601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c6b84-e7b8-4fb6-ab9c-310feca61323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:22.275205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-9e8c6b84-e7b8-4fb6-ab9c-310feca61323 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:08:22.275205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:22.275205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:22.275205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:22.275205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:22.275205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:22.275205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:22.275205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:22.275205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:22.275205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:22.275205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:22.275205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:22.275205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:22.275205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:22.275205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:22.275205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:22.276989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:22.276989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:22.276989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:22.276989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:22.276989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:22.276989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:22.276989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:22.276989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:22.276989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:22.276989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:22.276989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:22.276989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:22.276989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:22.276989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:22.276989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:22.276989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:22.278402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:22.278402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:22.278402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:22.278402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:22.278402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:22.278402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:22.278402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:22.278402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:22.278402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:22.278402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:22.278402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:08:22.278402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:08:22.279571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7a036-c3ec-4859-962e-fd83c81a510c None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:22.281725 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 156/303] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:22.284014 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c6b84-e7b8-4fb6-ab9c-310feca61323 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:22.355138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bfa53-c8b1-423b-9e00-beec9e483465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:22.356126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bfa53-c8b1-423b-9e00-beec9e483465 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:22.356422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bfa53-c8b1-423b-9e00-beec9e483465 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:22.356699 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bfa53-c8b1-423b-9e00-beec9e483465 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:22.357438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bfa53-c8b1-423b-9e00-beec9e483465 None admin] RBAC: Adding query filter params (name=member) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:22.357749 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bfa53-c8b1-423b-9e00-beec9e483465 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:22.359297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bfa53-c8b1-423b-9e00-beec9e483465 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:22.372327 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 148/304] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:22.373361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bfa53-c8b1-423b-9e00-beec9e483465 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:22.441356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:22.465122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:22.491283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c6b84-e7b8-4fb6-ab9c-310feca61323 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:22.553401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bfa53-c8b1-423b-9e00-beec9e483465 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:22.559030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5110fd-45cc-4ee1-b04c-8e42627e8601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:22.559030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5110fd-45cc-4ee1-b04c-8e42627e8601 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:22.559488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5110fd-45cc-4ee1-b04c-8e42627e8601 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:22.559622 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5110fd-45cc-4ee1-b04c-8e42627e8601 None admin] PATH_INFO: `/v3/groups` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:22.560280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5110fd-45cc-4ee1-b04c-8e42627e8601 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:22.561785 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5110fd-45cc-4ee1-b04c-8e42627e8601 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:22.618640 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f749950-bcb8-4791-bdc4-4eceb70c6bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:22.619847 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f749950-bcb8-4791-bdc4-4eceb70c6bc4 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:22.620345 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f749950-bcb8-4791-bdc4-4eceb70c6bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:22.620711 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f749950-bcb8-4791-bdc4-4eceb70c6bc4 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:22.621437 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f749950-bcb8-4791-bdc4-4eceb70c6bc4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:22.621746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f749950-bcb8-4791-bdc4-4eceb70c6bc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:22.623222 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f749950-bcb8-4791-bdc4-4eceb70c6bc4 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:22.638444 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 149/305] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:22.640189 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f749950-bcb8-4791-bdc4-4eceb70c6bc4 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:22.677792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 157/306] 10.222.0.62 () {60 vars in 1134 bytes} [Wed Sep 7 13:08:22 2022] POST /identity/v3/groups => generated 229 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:08:22.679645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5110fd-45cc-4ee1-b04c-8e42627e8601 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:22.835893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f749950-bcb8-4791-bdc4-4eceb70c6bc4 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:22.839440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:22.898160 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c9322-ed52-4c40-bedf-82b4ad2dbb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:22.898913 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c9322-ed52-4c40-bedf-82b4ad2dbb69 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:22.899232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c9322-ed52-4c40-bedf-82b4ad2dbb69 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:22.899510 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c9322-ed52-4c40-bedf-82b4ad2dbb69 None admin] PATH_INFO: `/v3/users/3dfbc9f1a02f4720aed732eb352dbb5e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:22.901013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c9322-ed52-4c40-bedf-82b4ad2dbb69 None admin] RBAC: Authorizing `identity:get_user(user_id=3dfbc9f1a02f4720aed732eb352dbb5e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:22.902749 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c9322-ed52-4c40-bedf-82b4ad2dbb69 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:22.904949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 150/307] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:08:22 2022] GET /identity/v3/users/3dfbc9f1a02f4720aed732eb352dbb5e => generated 293 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:22.911715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c9322-ed52-4c40-bedf-82b4ad2dbb69 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:22.923636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5110fd-45cc-4ee1-b04c-8e42627e8601 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:22.994481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b534fc-15a0-48ea-9332-9239fcff1d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:22.995539 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b534fc-15a0-48ea-9332-9239fcff1d25 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:22.995970 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b534fc-15a0-48ea-9332-9239fcff1d25 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:22.996371 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b534fc-15a0-48ea-9332-9239fcff1d25 None admin] PATH_INFO: `/v3/users/da94718d76c2418689b7ad9beb7bc01e` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:23.016113 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b534fc-15a0-48ea-9332-9239fcff1d25 None admin] RBAC: Authorizing `identity:get_user(user_id=da94718d76c2418689b7ad9beb7bc01e)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:23.017625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b534fc-15a0-48ea-9332-9239fcff1d25 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:23.019944 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 158/308] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:08:22 2022] GET /identity/v3/users/da94718d76c2418689b7ad9beb7bc01e => generated 293 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:23.027404 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b534fc-15a0-48ea-9332-9239fcff1d25 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:23.046161 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:23.053940 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c9322-ed52-4c40-bedf-82b4ad2dbb69 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:23.101941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f5e98-652d-408d-8d64-2fee0183bde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:23.102653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f5e98-652d-408d-8d64-2fee0183bde8 None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:23.102939 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f5e98-652d-408d-8d64-2fee0183bde8 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:23.103262 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f5e98-652d-408d-8d64-2fee0183bde8 None admin] PATH_INFO: `/v3/system/users/3dfbc9f1a02f4720aed732eb352dbb5e/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:23.104985 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f5e98-652d-408d-8d64-2fee0183bde8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3dfbc9f1a02f4720aed732eb352dbb5e, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:23.106662 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f5e98-652d-408d-8d64-2fee0183bde8 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:23.163197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b534fc-15a0-48ea-9332-9239fcff1d25 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:23.223787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253a829-984e-4415-b701-79d7e4166262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:23.224423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253a829-984e-4415-b701-79d7e4166262 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:23.224707 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253a829-984e-4415-b701-79d7e4166262 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:23.224989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253a829-984e-4415-b701-79d7e4166262 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:23.225800 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253a829-984e-4415-b701-79d7e4166262 None admin] RBAC: Adding query filter params (role.id=0503208661c44c4f8f52e30b4f21fff7, user.id=da94718d76c2418689b7ad9beb7bc01e, scope.system=all) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:23.226119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253a829-984e-4415-b701-79d7e4166262 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:23.227940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253a829-984e-4415-b701-79d7e4166262 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:23.241486 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 159/309] 10.222.0.62 () {56 vars in 1308 bytes} [Wed Sep 7 13:08:23 2022] GET /identity/v3/role_assignments?user.id=da94718d76c2418689b7ad9beb7bc01e&scope.system=all&role.id=0503208661c44c4f8f52e30b4f21fff7 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:08:23.245751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 151/310] 10.222.0.62 () {58 vars in 1334 bytes} [Wed Sep 7 13:08:22 2022] PUT /identity/v3/system/users/3dfbc9f1a02f4720aed732eb352dbb5e/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:23.351116 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881a241-5adc-4199-ab01-c65ec5b2bbd7 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:23.351615 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881a241-5adc-4199-ab01-c65ec5b2bbd7 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:23.351784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881a241-5adc-4199-ab01-c65ec5b2bbd7 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:23.354236 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 160/311] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:23.360903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157708a9-6466-49f5-bac6-b2375408e5ff None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:23.361720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157708a9-6466-49f5-bac6-b2375408e5ff None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:23.361720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157708a9-6466-49f5-bac6-b2375408e5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:23.391878 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-157708a9-6466-49f5-bac6-b2375408e5ff None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:23.393989 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-157708a9-6466-49f5-bac6-b2375408e5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:23.441898 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:23.538659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 152/312] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:24.067597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881a241-5adc-4199-ab01-c65ec5b2bbd7 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:24.228735 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:24.282534 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881a241-5adc-4199-ab01-c65ec5b2bbd7 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:24.354302 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32d1bc-b611-416d-bd07-7399bfc21740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:24.355323 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32d1bc-b611-416d-bd07-7399bfc21740 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:24.355806 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32d1bc-b611-416d-bd07-7399bfc21740 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:24.356326 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32d1bc-b611-416d-bd07-7399bfc21740 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:24.384397 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32d1bc-b611-416d-bd07-7399bfc21740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:24.395791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-dd32d1bc-b611-416d-bd07-7399bfc21740 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:08:24.395791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:24.395791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:24.395791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:24.395791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:24.395791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:24.395791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:24.395791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:24.395791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:24.395791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:24.395791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:24.395791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:24.395791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:24.395791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:24.395791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:24.395791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:24.396940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:24.396940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:24.396940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:24.396940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:24.396940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:24.396940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:24.396940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:24.396940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:24.396940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:24.396940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:24.396940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:24.396940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:24.396940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:24.396940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:24.396940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:24.396940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:24.397764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:24.397764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:24.397764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:24.397764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:24.397764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:24.397764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:24.397764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:24.397764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:24.397764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:24.397764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:24.397764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:08:24.397764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:08:24.398373 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 161/313] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:24.407665 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157708a9-6466-49f5-bac6-b2375408e5ff None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:24.549728 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157708a9-6466-49f5-bac6-b2375408e5ff None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:24.599472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d56e97-867b-4da2-96ac-fd095ed2e0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:24.600759 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d56e97-867b-4da2-96ac-fd095ed2e0a9 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:24.601538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d56e97-867b-4da2-96ac-fd095ed2e0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:24.602308 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d56e97-867b-4da2-96ac-fd095ed2e0a9 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:24.603902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d56e97-867b-4da2-96ac-fd095ed2e0a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:24.604932 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d56e97-867b-4da2-96ac-fd095ed2e0a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:24.607411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d56e97-867b-4da2-96ac-fd095ed2e0a9 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:24.624998 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 153/314] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:24.635770 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32d1bc-b611-416d-bd07-7399bfc21740 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:24.837847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32d1bc-b611-416d-bd07-7399bfc21740 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:24.888737 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254f0b1-d8fd-463e-999d-bc4b17b6379c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:24.889698 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254f0b1-d8fd-463e-999d-bc4b17b6379c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:24.890000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254f0b1-d8fd-463e-999d-bc4b17b6379c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:24.890274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254f0b1-d8fd-463e-999d-bc4b17b6379c None admin] PATH_INFO: `/v3/users/47fbab21dec64e58b748e24cc8031e4f` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:24.892178 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254f0b1-d8fd-463e-999d-bc4b17b6379c None admin] RBAC: Authorizing `identity:get_user(user_id=47fbab21dec64e58b748e24cc8031e4f)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:24.894561 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254f0b1-d8fd-463e-999d-bc4b17b6379c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:24.897316 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 162/315] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:08:24 2022] GET /identity/v3/users/47fbab21dec64e58b748e24cc8031e4f => generated 297 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:24.905590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d56e97-867b-4da2-96ac-fd095ed2e0a9 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:24.951812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:25.092236 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d56e97-867b-4da2-96ac-fd095ed2e0a9 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:25.161689 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbb57b-e605-48f2-a846-163e5f079fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:25.162652 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbb57b-e605-48f2-a846-163e5f079fbc None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:25.163193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbb57b-e605-48f2-a846-163e5f079fbc None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:25.163685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbb57b-e605-48f2-a846-163e5f079fbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:25.298141 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbb57b-e605-48f2-a846-163e5f079fbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:25.301134 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbb57b-e605-48f2-a846-163e5f079fbc None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:25.355343 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:25.501330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 154/316] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:08:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:08:25.511918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254f0b1-d8fd-463e-999d-bc4b17b6379c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:25.518219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:25.714357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254f0b1-d8fd-463e-999d-bc4b17b6379c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:25.761036 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9cddf1-b5ad-4b8a-9c29-f162b972160d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:25.761630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cddf1-b5ad-4b8a-9c29-f162b972160d None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:25.761957 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cddf1-b5ad-4b8a-9c29-f162b972160d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:25.762234 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cddf1-b5ad-4b8a-9c29-f162b972160d None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:25.763658 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9cddf1-b5ad-4b8a-9c29-f162b972160d None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:25.765167 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9cddf1-b5ad-4b8a-9c29-f162b972160d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:25.771106 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 163/317] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:25 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:25.773956 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbb57b-e605-48f2-a846-163e5f079fbc None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:25.897224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae36a91-ef66-4c80-9e01-4ee96dbf470d None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:25.897551 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae36a91-ef66-4c80-9e01-4ee96dbf470d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:25.897784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae36a91-ef66-4c80-9e01-4ee96dbf470d None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:25.900292 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 164/318] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:25.909230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba279c16-10c7-4d25-9713-667b57f4f32b None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:25.909847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba279c16-10c7-4d25-9713-667b57f4f32b None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:25.910362 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba279c16-10c7-4d25-9713-667b57f4f32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:25.923960 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbb57b-e605-48f2-a846-163e5f079fbc None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:25.935822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-ba279c16-10c7-4d25-9713-667b57f4f32b None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:25.937441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ba279c16-10c7-4d25-9713-667b57f4f32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:25.969820 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:25.984584 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069946de-669b-4126-828a-10bd103d9a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:25.994105 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069946de-669b-4126-828a-10bd103d9a87 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:25.994105 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069946de-669b-4126-828a-10bd103d9a87 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:25.994105 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069946de-669b-4126-828a-10bd103d9a87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:25.994105 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069946de-669b-4126-828a-10bd103d9a87 None admin] RBAC: Adding query filter params (role.id=0503208661c44c4f8f52e30b4f21fff7, scope.project.id=920dfa8ad35b4a429aea367ae3067b8e, user.id=47fbab21dec64e58b748e24cc8031e4f) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:25.994105 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069946de-669b-4126-828a-10bd103d9a87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:25.994838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069946de-669b-4126-828a-10bd103d9a87 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:26.011270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 155/319] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:08:25 2022] GET /identity/v3/role_assignments?user.id=47fbab21dec64e58b748e24cc8031e4f&scope.project.id=920dfa8ad35b4a429aea367ae3067b8e&role.id=0503208661c44c4f8f52e30b4f21fff7 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:26.023046 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:26.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 165/320] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:26.339960 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069946de-669b-4126-828a-10bd103d9a87 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:26.549975 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be18b6-5444-4a15-8a94-76be2f23e136 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:26.550779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be18b6-5444-4a15-8a94-76be2f23e136 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:26.550915 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be18b6-5444-4a15-8a94-76be2f23e136 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:26.553669 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 166/321] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:26.559158 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069946de-669b-4126-828a-10bd103d9a87 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:26.562490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44397a0e-41d3-4e2b-a4fb-6e7ae8d0c0ce None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:26.562895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44397a0e-41d3-4e2b-a4fb-6e7ae8d0c0ce None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:26.563287 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44397a0e-41d3-4e2b-a4fb-6e7ae8d0c0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:26.593776 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-44397a0e-41d3-4e2b-a4fb-6e7ae8d0c0ce None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:26.597635 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-44397a0e-41d3-4e2b-a4fb-6e7ae8d0c0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:26.633856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4dbf8-49a2-4e5f-b5f7-6be68247c706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:26.634934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4dbf8-49a2-4e5f-b5f7-6be68247c706 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:26.635539 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4dbf8-49a2-4e5f-b5f7-6be68247c706 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:26.636256 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4dbf8-49a2-4e5f-b5f7-6be68247c706 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:26.638080 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4dbf8-49a2-4e5f-b5f7-6be68247c706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:26.640802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4dbf8-49a2-4e5f-b5f7-6be68247c706 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:26.645104 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 156/322] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:08:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:26.649983 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4dbf8-49a2-4e5f-b5f7-6be68247c706 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:26.743139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 167/323] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:08:26.759359 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:26.813110 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4dbf8-49a2-4e5f-b5f7-6be68247c706 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:26.854272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f58464-86cc-4f18-a668-36a730c86389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:26.854925 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f58464-86cc-4f18-a668-36a730c86389 None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:26.855222 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f58464-86cc-4f18-a668-36a730c86389 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:26.855494 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f58464-86cc-4f18-a668-36a730c86389 None admin] PATH_INFO: `/v3/groups` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:26.856151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f58464-86cc-4f18-a668-36a730c86389 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:26.857580 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f58464-86cc-4f18-a668-36a730c86389 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:26.896596 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb1107-f8b6-4fd3-bfd5-3613af8a6f9e None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:26.896970 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb1107-f8b6-4fd3-bfd5-3613af8a6f9e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:26.897428 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb1107-f8b6-4fd3-bfd5-3613af8a6f9e None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:26.899069 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 168/324] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:08:26.907460 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987f9ad-ba4a-4b6c-823a-939411e7cba1 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:26.907839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987f9ad-ba4a-4b6c-823a-939411e7cba1 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:26.908168 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987f9ad-ba4a-4b6c-823a-939411e7cba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:26.932271 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-4987f9ad-ba4a-4b6c-823a-939411e7cba1 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:26.934343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-4987f9ad-ba4a-4b6c-823a-939411e7cba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:26.993326 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 157/325] 10.222.0.62 () {60 vars in 1134 bytes} [Wed Sep 7 13:08:26 2022] POST /identity/v3/groups => generated 226 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:08:27.059727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f58464-86cc-4f18-a668-36a730c86389 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:27.064103 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 169/326] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:08:27.211547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f58464-86cc-4f18-a668-36a730c86389 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:27.235881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:27.260656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eaab8d-6f2f-4d08-aa1d-1eea0ea7dfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:27.261809 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eaab8d-6f2f-4d08-aa1d-1eea0ea7dfef None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:27.262321 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eaab8d-6f2f-4d08-aa1d-1eea0ea7dfef None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:27.262821 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eaab8d-6f2f-4d08-aa1d-1eea0ea7dfef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:27.285346 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eaab8d-6f2f-4d08-aa1d-1eea0ea7dfef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:27.291622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-d5eaab8d-6f2f-4d08-aa1d-1eea0ea7dfef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:08:27.291622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:27.291622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:27.291622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:27.291622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:27.291622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:27.291622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:27.291622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:27.291622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:27.291622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:27.291622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:27.291622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:27.291622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:27.291622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:27.291622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:27.291622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:27.296341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:27.296341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:27.296341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:27.296341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:27.296341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:27.296341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:27.296341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:27.296341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:27.296341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:27.296341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:27.296341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:27.296341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:27.296341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:27.296341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:27.296341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:27.296341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:27.297185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:27.297185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:27.297185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:27.297185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:27.297185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:27.297185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:27.297185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:27.297185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:27.297185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:27.297185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:27.297185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:08:27.297185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:27.297185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 158/327] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:08:27.300546 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987f9ad-ba4a-4b6c-823a-939411e7cba1 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:27.386951 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:27.392518 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eaab8d-6f2f-4d08-aa1d-1eea0ea7dfef None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:27.477153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987f9ad-ba4a-4b6c-823a-939411e7cba1 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:27.549876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77dd24-debb-4f85-92e4-df1d25254b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:27.551778 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77dd24-debb-4f85-92e4-df1d25254b78 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:27.551778 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77dd24-debb-4f85-92e4-df1d25254b78 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:27.552026 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77dd24-debb-4f85-92e4-df1d25254b78 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:27.553115 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77dd24-debb-4f85-92e4-df1d25254b78 None admin] RBAC: Adding query filter params (name=reader) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:27.553680 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77dd24-debb-4f85-92e4-df1d25254b78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:27.559055 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77dd24-debb-4f85-92e4-df1d25254b78 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:27.583176 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 170/328] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:27.586316 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77dd24-debb-4f85-92e4-df1d25254b78 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:27.613453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eaab8d-6f2f-4d08-aa1d-1eea0ea7dfef None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:27.701160 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fb19f-6d92-4b93-b3e4-0ebc525aeb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:27.702267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fb19f-6d92-4b93-b3e4-0ebc525aeb50 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:27.702736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fb19f-6d92-4b93-b3e4-0ebc525aeb50 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:27.703224 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fb19f-6d92-4b93-b3e4-0ebc525aeb50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:27.728785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fb19f-6d92-4b93-b3e4-0ebc525aeb50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:27.740121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-397fb19f-6d92-4b93-b3e4-0ebc525aeb50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:27.740121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:27.740121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:27.740121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:27.740121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:27.740121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:27.740121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:27.740121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:27.740121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:27.740121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:27.740121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:27.740121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:27.740121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:27.740121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:27.740121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:27.740121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:27.742023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:27.742023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:27.742023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:27.742023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:27.742023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:27.742023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:27.742023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:27.742023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:27.742023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:27.742023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:27.742023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:27.742023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:27.742023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:27.742023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:27.742023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:27.742023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:27.748856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:27.748856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:27.748856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:27.748856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:27.748856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:27.748856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:27.748856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:27.748856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:27.748856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:27.748856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:27.748856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:27.748856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:27.748856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 159/329] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:08:27.752101 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fb19f-6d92-4b93-b3e4-0ebc525aeb50 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:27.873277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:27.880891 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77dd24-debb-4f85-92e4-df1d25254b78 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:27.930359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:27.952418 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07991f6d-808e-4f21-997a-24c690e9e00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:27.953335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07991f6d-808e-4f21-997a-24c690e9e00c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:27.953645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07991f6d-808e-4f21-997a-24c690e9e00c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:27.954081 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07991f6d-808e-4f21-997a-24c690e9e00c None admin] PATH_INFO: `/v3/users/da94718d76c2418689b7ad9beb7bc01e` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:27.957381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07991f6d-808e-4f21-997a-24c690e9e00c None admin] RBAC: Authorizing `identity:get_user(user_id=da94718d76c2418689b7ad9beb7bc01e)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:27.959452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07991f6d-808e-4f21-997a-24c690e9e00c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:27.963812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 171/330] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:08:27 2022] GET /identity/v3/users/da94718d76c2418689b7ad9beb7bc01e => generated 293 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:08:27.972909 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07991f6d-808e-4f21-997a-24c690e9e00c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:28.007591 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fb19f-6d92-4b93-b3e4-0ebc525aeb50 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:28.099149 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcf25f-8ea7-4d97-b71c-cf9debc8fd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:28.100206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcf25f-8ea7-4d97-b71c-cf9debc8fd82 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:28.100700 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcf25f-8ea7-4d97-b71c-cf9debc8fd82 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:28.101190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcf25f-8ea7-4d97-b71c-cf9debc8fd82 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:28.102682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcf25f-8ea7-4d97-b71c-cf9debc8fd82 None admin] RBAC: Adding query filter params (name=member) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:28.103269 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcf25f-8ea7-4d97-b71c-cf9debc8fd82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:28.105778 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcf25f-8ea7-4d97-b71c-cf9debc8fd82 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:28.126615 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 160/331] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:28.139341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcf25f-8ea7-4d97-b71c-cf9debc8fd82 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:28.167607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07991f6d-808e-4f21-997a-24c690e9e00c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:28.243399 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c0043-b4a7-4fe4-97fe-0c82ddcf2378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:28.244701 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c0043-b4a7-4fe4-97fe-0c82ddcf2378 None admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:28.245342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c0043-b4a7-4fe4-97fe-0c82ddcf2378 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:28.246294 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c0043-b4a7-4fe4-97fe-0c82ddcf2378 None admin] PATH_INFO: `/v3/system/users/da94718d76c2418689b7ad9beb7bc01e/roles/0503208661c44c4f8f52e30b4f21fff7` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:28.248950 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c0043-b4a7-4fe4-97fe-0c82ddcf2378 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=da94718d76c2418689b7ad9beb7bc01e, role_id=0503208661c44c4f8f52e30b4f21fff7)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:28.250739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c0043-b4a7-4fe4-97fe-0c82ddcf2378 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:28.325489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 172/332] 10.222.0.62 () {58 vars in 1334 bytes} [Wed Sep 7 13:08:27 2022] PUT /identity/v3/system/users/da94718d76c2418689b7ad9beb7bc01e/roles/0503208661c44c4f8f52e30b4f21fff7 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:08:28.347246 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:28.353433 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcf25f-8ea7-4d97-b71c-cf9debc8fd82 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:28.436573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47ca6d-7004-4529-ad97-63c0e6ec9c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:28.437640 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47ca6d-7004-4529-ad97-63c0e6ec9c75 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:28.438114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47ca6d-7004-4529-ad97-63c0e6ec9c75 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:28.438588 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47ca6d-7004-4529-ad97-63c0e6ec9c75 None admin] PATH_INFO: `/v3/users/3dfbc9f1a02f4720aed732eb352dbb5e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:28.441373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47ca6d-7004-4529-ad97-63c0e6ec9c75 None admin] RBAC: Authorizing `identity:get_user(user_id=3dfbc9f1a02f4720aed732eb352dbb5e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:28.444305 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47ca6d-7004-4529-ad97-63c0e6ec9c75 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:28.447854 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 161/333] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:08:28 2022] GET /identity/v3/users/3dfbc9f1a02f4720aed732eb352dbb5e => generated 293 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:08:28.454199 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c0043-b4a7-4fe4-97fe-0c82ddcf2378 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:28.591681 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:28.719024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c0043-b4a7-4fe4-97fe-0c82ddcf2378 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:28.797868 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85f069-e172-4b12-8f60-ce584e000c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:28.799651 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85f069-e172-4b12-8f60-ce584e000c2a None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:28.800431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85f069-e172-4b12-8f60-ce584e000c2a None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:28.801160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85f069-e172-4b12-8f60-ce584e000c2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:28.802780 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85f069-e172-4b12-8f60-ce584e000c2a None admin] RBAC: Adding query filter params (role.id=bcc52f73cf03442e922a4757908334de, user.id=3dfbc9f1a02f4720aed732eb352dbb5e, scope.system=all) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:28.803536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85f069-e172-4b12-8f60-ce584e000c2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:28.806625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85f069-e172-4b12-8f60-ce584e000c2a None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:28.838918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 173/334] 10.222.0.62 () {56 vars in 1308 bytes} [Wed Sep 7 13:08:28 2022] GET /identity/v3/role_assignments?user.id=3dfbc9f1a02f4720aed732eb352dbb5e&scope.system=all&role.id=bcc52f73cf03442e922a4757908334de => generated 510 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:31.070344 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dec7f3-b655-40d5-bbf0-d46cd8794a57 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:31.071893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dec7f3-b655-40d5-bbf0-d46cd8794a57 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:31.072599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dec7f3-b655-40d5-bbf0-d46cd8794a57 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:31.075457 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 162/335] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:31.085305 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a278e68-1c16-4a53-9fd0-f17985f003a4 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:31.086091 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a278e68-1c16-4a53-9fd0-f17985f003a4 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:31.086335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a278e68-1c16-4a53-9fd0-f17985f003a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:31.127976 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-0a278e68-1c16-4a53-9fd0-f17985f003a4 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:31.130620 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0a278e68-1c16-4a53-9fd0-f17985f003a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:31.260969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 174/336] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:31.904032 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19662bef-8583-4b4c-b60b-0d5531f5715d None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:31.904856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19662bef-8583-4b4c-b60b-0d5531f5715d None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:31.905445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19662bef-8583-4b4c-b60b-0d5531f5715d None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:31.907860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 163/337] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:31.916685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc360a64-7ec7-4d8a-a577-1a10bbe085a7 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:31.917427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc360a64-7ec7-4d8a-a577-1a10bbe085a7 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:31.917967 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc360a64-7ec7-4d8a-a577-1a10bbe085a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:31.944128 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-bc360a64-7ec7-4d8a-a577-1a10bbe085a7 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:31.946292 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-bc360a64-7ec7-4d8a-a577-1a10bbe085a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:32.006434 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:32.056390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 175/338] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:32.255300 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19662bef-8583-4b4c-b60b-0d5531f5715d None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:32.503030 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19662bef-8583-4b4c-b60b-0d5531f5715d None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:32.559259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ffd4f-55e4-4e51-8396-1c3a4363cd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:32.559933 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ffd4f-55e4-4e51-8396-1c3a4363cd2d None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:32.560276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ffd4f-55e4-4e51-8396-1c3a4363cd2d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:32.560557 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ffd4f-55e4-4e51-8396-1c3a4363cd2d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:32.585866 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ffd4f-55e4-4e51-8396-1c3a4363cd2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:32.603401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-2c2ffd4f-55e4-4e51-8396-1c3a4363cd2d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:08:32.603401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:32.603401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:32.603401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:32.603401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:32.603401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:32.603401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:32.603401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:32.603401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:32.603401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:32.603401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:32.603401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:32.603401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:32.603401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:32.603401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:32.603401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:32.606078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:32.606078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:32.606078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:32.606078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:32.606078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:32.606078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:32.606078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:32.606078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:32.606078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:32.606078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:32.606078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:32.606078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:32.606078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:32.606078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:32.606078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:32.606078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:32.609369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:32.609369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:32.609369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:32.609369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:32.609369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:32.609369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:32.609369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:32.609369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:32.609369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:32.609369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:32.609369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:08:32.609369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:32.616819 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f999df5-31dd-4bfe-a154-cfd62ad3571e None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:32.616819 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f999df5-31dd-4bfe-a154-cfd62ad3571e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:32.616819 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f999df5-31dd-4bfe-a154-cfd62ad3571e None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:32.617928 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 164/339] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:08:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:32.617928 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ffd4f-55e4-4e51-8396-1c3a4363cd2d None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:32.619803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 176/340] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:32 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:32.637503 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2f580-fd08-4d66-9d96-42732fb5619c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:32.637981 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2f580-fd08-4d66-9d96-42732fb5619c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:32.638473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2f580-fd08-4d66-9d96-42732fb5619c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:32.687604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:32.712852 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-d6b2f580-fd08-4d66-9d96-42732fb5619c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:32.715008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d6b2f580-fd08-4d66-9d96-42732fb5619c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:32.805389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ffd4f-55e4-4e51-8396-1c3a4363cd2d None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:32.894720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 177/341] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:32.896089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2f580-fd08-4d66-9d96-42732fb5619c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:32.918814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad3367-becd-4f8f-abe8-796705ec1256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:32.919552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad3367-becd-4f8f-abe8-796705ec1256 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:32.919863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad3367-becd-4f8f-abe8-796705ec1256 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:32.920239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad3367-becd-4f8f-abe8-796705ec1256 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:32.920939 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad3367-becd-4f8f-abe8-796705ec1256 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:32.921250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad3367-becd-4f8f-abe8-796705ec1256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:32.922946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad3367-becd-4f8f-abe8-796705ec1256 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:32.947451 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 165/342] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:08:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:32.953889 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad3367-becd-4f8f-abe8-796705ec1256 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:33.161053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:33.173147 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad3367-becd-4f8f-abe8-796705ec1256 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:33.239648 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab51549-b80f-493b-86a7-911f3d8a680c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:33.240285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab51549-b80f-493b-86a7-911f3d8a680c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:33.240623 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab51549-b80f-493b-86a7-911f3d8a680c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:33.240915 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab51549-b80f-493b-86a7-911f3d8a680c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:33.267168 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2f580-fd08-4d66-9d96-42732fb5619c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:33.274002 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:33.429274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e75b7-a19d-4775-8850-010a598bb1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:33.430290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e75b7-a19d-4775-8850-010a598bb1e8 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:33.430771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e75b7-a19d-4775-8850-010a598bb1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:33.431286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e75b7-a19d-4775-8850-010a598bb1e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:33.432728 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab51549-b80f-493b-86a7-911f3d8a680c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:33.435667 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab51549-b80f-493b-86a7-911f3d8a680c None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:33.496203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e75b7-a19d-4775-8850-010a598bb1e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:33.506444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-3d4e75b7-a19d-4775-8850-010a598bb1e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:33.506444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:33.506444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:33.506444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:33.506444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:33.506444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:33.506444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:33.506444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:33.506444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:33.506444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:33.506444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:33.506444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:33.506444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:33.506444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:33.506444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:33.506444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:33.508320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:33.508320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:33.508320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:33.508320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:33.508320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:33.508320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:33.508320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:33.508320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:33.508320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:33.508320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:33.508320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:33.508320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:33.508320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:33.508320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:33.508320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:33.508320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:33.509804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:33.509804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:33.509804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:33.509804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:33.509804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:33.509804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:33.509804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:33.509804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:33.509804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:33.509804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:33.509804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:33.509804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:08:33.509804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 178/343] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 614 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:33.521265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e75b7-a19d-4775-8850-010a598bb1e8 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:33.845250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:33.882533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 166/344] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:08:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 929 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:08:33.884969 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436a4a0-c6be-44c4-9a82-cdc4b0d71f44 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:33.885469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436a4a0-c6be-44c4-9a82-cdc4b0d71f44 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:33.885940 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436a4a0-c6be-44c4-9a82-cdc4b0d71f44 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:33.892067 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 167/345] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:33.893631 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436a4a0-c6be-44c4-9a82-cdc4b0d71f44 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:33.916124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e75b7-a19d-4775-8850-010a598bb1e8 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:34.001193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b357e-8699-4004-89e0-3682e4c58d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:34.003240 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b357e-8699-4004-89e0-3682e4c58d4b None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:34.004043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b357e-8699-4004-89e0-3682e4c58d4b None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:34.004795 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b357e-8699-4004-89e0-3682e4c58d4b None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:34.006329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b357e-8699-4004-89e0-3682e4c58d4b None admin] RBAC: Adding query filter params (name=member) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:34.008847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b357e-8699-4004-89e0-3682e4c58d4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:34.012216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b357e-8699-4004-89e0-3682e4c58d4b None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:34.030515 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 179/346] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 511 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:08:34.033864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ec444-029c-4564-8fae-5a016a63789c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:34.034746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ec444-029c-4564-8fae-5a016a63789c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:34.035552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ec444-029c-4564-8fae-5a016a63789c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:34.077004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-eb7ec444-029c-4564-8fae-5a016a63789c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:34.080786 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-eb7ec444-029c-4564-8fae-5a016a63789c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:34.133014 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:34.159874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436a4a0-c6be-44c4-9a82-cdc4b0d71f44 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:34.216921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dae26-be19-4aec-8e59-6e4a23b0956a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:34.218150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dae26-be19-4aec-8e59-6e4a23b0956a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:34.218701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dae26-be19-4aec-8e59-6e4a23b0956a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:34.219222 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dae26-be19-4aec-8e59-6e4a23b0956a None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:34.221317 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dae26-be19-4aec-8e59-6e4a23b0956a None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:34.223542 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dae26-be19-4aec-8e59-6e4a23b0956a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:34.225804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 168/347] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:33 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:34.227013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dae26-be19-4aec-8e59-6e4a23b0956a None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:34.270938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 180/348] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:34.277543 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ec444-029c-4564-8fae-5a016a63789c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:34.322031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:34.416125 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dae26-be19-4aec-8e59-6e4a23b0956a None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:34.485455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a590c-4fe1-49ad-b017-2ca2efeebe15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:34.487130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a590c-4fe1-49ad-b017-2ca2efeebe15 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:34.487739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a590c-4fe1-49ad-b017-2ca2efeebe15 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:34.488237 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a590c-4fe1-49ad-b017-2ca2efeebe15 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:34.504791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a590c-4fe1-49ad-b017-2ca2efeebe15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:34.510407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-ab9a590c-4fe1-49ad-b017-2ca2efeebe15 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:34.510407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:34.510407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:34.510407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:34.510407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:34.510407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:34.510407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:34.510407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:34.510407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:34.510407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:34.510407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:34.510407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:34.510407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:34.510407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:34.510407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:34.510407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:34.511472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:34.511472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:34.511472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:34.511472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:34.511472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:34.511472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:34.511472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:34.511472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:34.511472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:34.511472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:34.511472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:34.511472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:34.511472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:34.511472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:34.511472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:34.511472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:34.512448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:34.512448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:34.512448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:34.512448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:34.512448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:34.512448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:34.512448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:34.512448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:34.512448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:34.512448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:34.512448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:34.512448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:34.515404 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 169/349] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:34.516648 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a590c-4fe1-49ad-b017-2ca2efeebe15 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:34.531814 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ec444-029c-4564-8fae-5a016a63789c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:34.620562 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93aad80-1257-4fca-9dfc-b5721ed5e7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:34.622346 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93aad80-1257-4fca-9dfc-b5721ed5e7fc None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:34.623198 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93aad80-1257-4fca-9dfc-b5721ed5e7fc None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:34.623874 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93aad80-1257-4fca-9dfc-b5721ed5e7fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:34.686177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:34.737439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a590c-4fe1-49ad-b017-2ca2efeebe15 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:34.777024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:34.779019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93aad80-1257-4fca-9dfc-b5721ed5e7fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:34.782419 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93aad80-1257-4fca-9dfc-b5721ed5e7fc None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:34.812217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d498ecb-3a42-4ad3-95ce-995bddd36a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:34.813015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d498ecb-3a42-4ad3-95ce-995bddd36a0e None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:34.813478 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d498ecb-3a42-4ad3-95ce-995bddd36a0e None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:34.813978 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d498ecb-3a42-4ad3-95ce-995bddd36a0e None admin] PATH_INFO: `/v3/groups/59e391a4350244c9888947a0896ee4e2` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:34.824877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d498ecb-3a42-4ad3-95ce-995bddd36a0e None admin] RBAC: Authorizing `identity:get_group(group_id=59e391a4350244c9888947a0896ee4e2)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:34.826810 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d498ecb-3a42-4ad3-95ce-995bddd36a0e None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:34.828911 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 170/350] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 7 13:08:34 2022] GET /identity/v3/groups/59e391a4350244c9888947a0896ee4e2 => generated 226 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:34.835839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527baaa-e027-4111-88da-4b61d2757029 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:34.836389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527baaa-e027-4111-88da-4b61d2757029 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:34.837301 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527baaa-e027-4111-88da-4b61d2757029 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:34.843179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 171/351] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:34 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:34.845433 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527baaa-e027-4111-88da-4b61d2757029 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:34.987008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 181/352] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:08:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 710 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:08:34.988881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88aae38-d5ea-43f2-bcc7-d39980ebac79 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:34.989318 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88aae38-d5ea-43f2-bcc7-d39980ebac79 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:34.989666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88aae38-d5ea-43f2-bcc7-d39980ebac79 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:34.991261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 182/353] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:34.992623 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88aae38-d5ea-43f2-bcc7-d39980ebac79 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:35.042080 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527baaa-e027-4111-88da-4b61d2757029 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:35.104169 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e0068e-1a6d-47df-8dd8-8ef42e2d7292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:35.105705 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0068e-1a6d-47df-8dd8-8ef42e2d7292 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:35.106184 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0068e-1a6d-47df-8dd8-8ef42e2d7292 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:35.106658 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0068e-1a6d-47df-8dd8-8ef42e2d7292 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:35.107928 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e0068e-1a6d-47df-8dd8-8ef42e2d7292 None admin] RBAC: Adding query filter params (name=member) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:35.108587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e0068e-1a6d-47df-8dd8-8ef42e2d7292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:35.111927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e0068e-1a6d-47df-8dd8-8ef42e2d7292 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:35.114642 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:35.127520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 172/354] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:35.133794 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e0068e-1a6d-47df-8dd8-8ef42e2d7292 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:35.194341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88aae38-d5ea-43f2-bcc7-d39980ebac79 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:35.215770 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:35.257421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34122e3b-0dd1-4b93-a5a4-cfb70c733131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:35.258419 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34122e3b-0dd1-4b93-a5a4-cfb70c733131 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:35.258905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34122e3b-0dd1-4b93-a5a4-cfb70c733131 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:35.259355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34122e3b-0dd1-4b93-a5a4-cfb70c733131 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:35.282251 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34122e3b-0dd1-4b93-a5a4-cfb70c733131 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:35.294616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-34122e3b-0dd1-4b93-a5a4-cfb70c733131 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:08:35.294616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:35.294616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:35.294616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:35.294616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:35.294616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:35.294616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:35.294616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:35.294616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:35.294616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:35.294616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:35.294616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:35.294616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:35.294616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:35.294616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:35.294616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:35.295864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:35.295864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:35.295864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:35.295864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:35.295864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:35.295864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:35.295864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:35.295864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:35.295864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:35.295864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:35.295864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:35.295864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:35.295864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:35.295864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:35.295864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:35.295864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:35.296794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:35.296794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:35.296794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:35.296794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:35.296794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:35.296794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:35.296794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:35.296794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:35.296794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:35.296794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:35.296794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:08:35.296794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:08:35.296794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 183/355] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:08:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:35.305328 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a8516-475c-4b8d-b8a2-a74e28a5245f None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:35.305328 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a8516-475c-4b8d-b8a2-a74e28a5245f None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:35.305328 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a8516-475c-4b8d-b8a2-a74e28a5245f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:35.334747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-696a8516-475c-4b8d-b8a2-a74e28a5245f None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:35.335035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e0068e-1a6d-47df-8dd8-8ef42e2d7292 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:35.336605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-696a8516-475c-4b8d-b8a2-a74e28a5245f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:35.414074 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329caa2b-e6b4-4316-8818-1fbaacfc5ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:35.415271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329caa2b-e6b4-4316-8818-1fbaacfc5ab0 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:35.415785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329caa2b-e6b4-4316-8818-1fbaacfc5ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:35.416285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329caa2b-e6b4-4316-8818-1fbaacfc5ab0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:35.418642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329caa2b-e6b4-4316-8818-1fbaacfc5ab0 None admin] RBAC: Adding query filter params (group.id=59e391a4350244c9888947a0896ee4e2, role.id=fc0a7877dd0f4903a2a6fcc01a829c6a, scope.project.id=1fcdd13492724455ae65d6e80d2f41bc) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:35.419282 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329caa2b-e6b4-4316-8818-1fbaacfc5ab0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:35.423540 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329caa2b-e6b4-4316-8818-1fbaacfc5ab0 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:35.437109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 173/356] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 7 13:08:35 2022] GET /identity/v3/role_assignments?group.id=59e391a4350244c9888947a0896ee4e2&scope.project.id=1fcdd13492724455ae65d6e80d2f41bc&role.id=fc0a7877dd0f4903a2a6fcc01a829c6a => generated 260 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:35.438638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329caa2b-e6b4-4316-8818-1fbaacfc5ab0 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:35.472565 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 184/357] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:35.473892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac22b9-d555-44c2-969a-79b90091589a None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:35.474737 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac22b9-d555-44c2-969a-79b90091589a None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:35.475233 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac22b9-d555-44c2-969a-79b90091589a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:35.501654 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-23ac22b9-d555-44c2-969a-79b90091589a None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:35.505255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-23ac22b9-d555-44c2-969a-79b90091589a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:35.595289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:35.655068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329caa2b-e6b4-4316-8818-1fbaacfc5ab0 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:35.670397 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 185/358] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:35.671672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac22b9-d555-44c2-969a-79b90091589a None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:35.682078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:35.746892 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af55aa-5d6c-47e4-9287-988d02efbc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:35.748425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af55aa-5d6c-47e4-9287-988d02efbc82 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:35.749134 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af55aa-5d6c-47e4-9287-988d02efbc82 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:35.749896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af55aa-5d6c-47e4-9287-988d02efbc82 None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:35.753018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af55aa-5d6c-47e4-9287-988d02efbc82 None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:35.757016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af55aa-5d6c-47e4-9287-988d02efbc82 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:35.761999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 174/359] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:35 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:35.763774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af55aa-5d6c-47e4-9287-988d02efbc82 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:35.911762 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac22b9-d555-44c2-969a-79b90091589a None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:35.979492 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43e3f4-3e65-4037-bb4c-64ccc4f1bc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:35.980586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43e3f4-3e65-4037-bb4c-64ccc4f1bc0f None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:35.981030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43e3f4-3e65-4037-bb4c-64ccc4f1bc0f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:35.981502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43e3f4-3e65-4037-bb4c-64ccc4f1bc0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:35.986239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af55aa-5d6c-47e4-9287-988d02efbc82 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:36.051538 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:36.054167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe2db7-4443-4911-9f50-b5ac57b2acf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:36.055318 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe2db7-4443-4911-9f50-b5ac57b2acf9 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:36.055780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe2db7-4443-4911-9f50-b5ac57b2acf9 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:36.056199 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe2db7-4443-4911-9f50-b5ac57b2acf9 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:36.057139 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe2db7-4443-4911-9f50-b5ac57b2acf9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:36.057604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe2db7-4443-4911-9f50-b5ac57b2acf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:36.059284 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe2db7-4443-4911-9f50-b5ac57b2acf9 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:36.075433 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 175/360] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:08:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:36.077375 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe2db7-4443-4911-9f50-b5ac57b2acf9 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:36.096172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43e3f4-3e65-4037-bb4c-64ccc4f1bc0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:36.100338 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43e3f4-3e65-4037-bb4c-64ccc4f1bc0f None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:36.142885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:36.274821 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe2db7-4443-4911-9f50-b5ac57b2acf9 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:36.283793 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 186/361] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:08:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:08:36.285520 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43e3f4-3e65-4037-bb4c-64ccc4f1bc0f None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:36.365831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bae83-76ae-4728-8f74-7819758e76e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:36.367260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bae83-76ae-4728-8f74-7819758e76e7 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:36.367909 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bae83-76ae-4728-8f74-7819758e76e7 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:36.368547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bae83-76ae-4728-8f74-7819758e76e7 None admin] PATH_INFO: `/v3/groups/59e391a4350244c9888947a0896ee4e2` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:36.370911 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bae83-76ae-4728-8f74-7819758e76e7 None admin] RBAC: Authorizing `identity:get_group(group_id=59e391a4350244c9888947a0896ee4e2)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:36.374518 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bae83-76ae-4728-8f74-7819758e76e7 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:36.379488 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 176/362] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 7 13:08:36 2022] GET /identity/v3/groups/59e391a4350244c9888947a0896ee4e2 => generated 226 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:36.382106 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bae83-76ae-4728-8f74-7819758e76e7 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:36.452000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:36.513736 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43e3f4-3e65-4037-bb4c-64ccc4f1bc0f None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:36.590665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176326d-e71a-4874-8804-44bc88b042f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:36.591748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176326d-e71a-4874-8804-44bc88b042f1 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:36.592241 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176326d-e71a-4874-8804-44bc88b042f1 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:36.592728 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176326d-e71a-4874-8804-44bc88b042f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:36.621392 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bae83-76ae-4728-8f74-7819758e76e7 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:36.634964 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:36.696877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be078aac-4d74-499e-a893-f3411ca37d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:36.697894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be078aac-4d74-499e-a893-f3411ca37d6d None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:36.697894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be078aac-4d74-499e-a893-f3411ca37d6d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:36.698167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be078aac-4d74-499e-a893-f3411ca37d6d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:36.717068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be078aac-4d74-499e-a893-f3411ca37d6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:36.730425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-be078aac-4d74-499e-a893-f3411ca37d6d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:08:36.730425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:36.730425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:36.730425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:36.730425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:36.730425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:36.730425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:36.730425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:36.730425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:36.730425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:36.730425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:36.730425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:36.730425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:36.730425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:36.730425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:36.730425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:36.732096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:36.732096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:36.732096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:36.732096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:36.732096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:36.732096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:36.732096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:36.732096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:36.732096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:36.732096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:36.732096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:36.732096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:36.732096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:36.732096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:36.732096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:36.732096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:36.735685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:36.735685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:36.735685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:36.735685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:36.735685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:36.735685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:36.735685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:36.735685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:36.735685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:36.735685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:36.735685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 13:08:36.735685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:36.735685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 177/363] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:36.735685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be078aac-4d74-499e-a893-f3411ca37d6d None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:36.743595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176326d-e71a-4874-8804-44bc88b042f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:36.746518 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176326d-e71a-4874-8804-44bc88b042f1 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:36.927582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:36.979764 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be078aac-4d74-499e-a893-f3411ca37d6d None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:36.991854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 187/364] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:08:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:08:36.993203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176326d-e71a-4874-8804-44bc88b042f1 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:37.056639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857084a9-9823-4600-ad03-3507248897c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:37.058165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857084a9-9823-4600-ad03-3507248897c0 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:37.058962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857084a9-9823-4600-ad03-3507248897c0 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:37.059738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857084a9-9823-4600-ad03-3507248897c0 None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:37.061852 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857084a9-9823-4600-ad03-3507248897c0 None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:37.065464 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857084a9-9823-4600-ad03-3507248897c0 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:37.076556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 178/365] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:36 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:37.076556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857084a9-9823-4600-ad03-3507248897c0 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:37.152714 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:37.214624 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176326d-e71a-4874-8804-44bc88b042f1 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:37.292398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849795e-eb62-492d-8268-a098bbb2cdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:37.293405 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849795e-eb62-492d-8268-a098bbb2cdfe None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:37.293899 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849795e-eb62-492d-8268-a098bbb2cdfe None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:37.294567 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849795e-eb62-492d-8268-a098bbb2cdfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:37.296082 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849795e-eb62-492d-8268-a098bbb2cdfe None admin] RBAC: Adding query filter params (group.id=59e391a4350244c9888947a0896ee4e2, role.id=bcc52f73cf03442e922a4757908334de, scope.project.id=920dfa8ad35b4a429aea367ae3067b8e) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:37.296623 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849795e-eb62-492d-8268-a098bbb2cdfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:37.299622 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849795e-eb62-492d-8268-a098bbb2cdfe None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:37.308689 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857084a9-9823-4600-ad03-3507248897c0 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:37.311601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 188/366] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 7 13:08:36 2022] GET /identity/v3/role_assignments?group.id=59e391a4350244c9888947a0896ee4e2&scope.project.id=920dfa8ad35b4a429aea367ae3067b8e&role.id=bcc52f73cf03442e922a4757908334de => generated 260 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:37.313195 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849795e-eb62-492d-8268-a098bbb2cdfe None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:37.403657 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9d6f9-e523-441d-9ddf-956525b2a17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:37.404842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9d6f9-e523-441d-9ddf-956525b2a17d None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:37.405338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9d6f9-e523-441d-9ddf-956525b2a17d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:37.405842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9d6f9-e523-441d-9ddf-956525b2a17d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:37.439164 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9d6f9-e523-441d-9ddf-956525b2a17d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:37.443575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:37.452548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-c0d9d6f9-e523-441d-9ddf-956525b2a17d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:08:37.452548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:37.452548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:37.452548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:37.452548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:37.452548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:37.452548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:37.452548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:37.452548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:37.452548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:37.452548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:37.452548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:37.452548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:37.452548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:37.452548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:37.452548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:37.454298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:37.454298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:37.454298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:37.454298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:37.454298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:37.454298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:37.454298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:37.454298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:37.454298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:37.454298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:37.454298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:37.454298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:37.454298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:37.454298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:37.454298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:37.454298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:37.456469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:37.456469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:37.456469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:37.456469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:37.456469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:37.456469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:37.456469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:37.456469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:37.456469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:37.456469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:37.456469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:08:37.456469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:37.457928 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 179/367] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:08:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:37.473880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9d6f9-e523-441d-9ddf-956525b2a17d None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:37.576065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849795e-eb62-492d-8268-a098bbb2cdfe None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:37.652107 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24e084-2172-441e-b2b3-1369df53ed1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:37.653179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24e084-2172-441e-b2b3-1369df53ed1a None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:37.653676 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24e084-2172-441e-b2b3-1369df53ed1a None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:37.654421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24e084-2172-441e-b2b3-1369df53ed1a None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:37.657299 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24e084-2172-441e-b2b3-1369df53ed1a None admin] RBAC: Adding query filter params (name=reader) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:37.657779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24e084-2172-441e-b2b3-1369df53ed1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:37.661582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24e084-2172-441e-b2b3-1369df53ed1a None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:37.679443 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 189/368] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:37.680722 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24e084-2172-441e-b2b3-1369df53ed1a None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:37.709505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:37.722303 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9d6f9-e523-441d-9ddf-956525b2a17d None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:37.819253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409cc628-b0a0-406e-8a76-0fc41cc62d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:37.820856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409cc628-b0a0-406e-8a76-0fc41cc62d6c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:37.821316 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409cc628-b0a0-406e-8a76-0fc41cc62d6c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:37.821776 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409cc628-b0a0-406e-8a76-0fc41cc62d6c None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:37.824307 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409cc628-b0a0-406e-8a76-0fc41cc62d6c None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:37.827440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409cc628-b0a0-406e-8a76-0fc41cc62d6c None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:37.831592 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 180/369] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:37 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:37.833470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409cc628-b0a0-406e-8a76-0fc41cc62d6c None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:37.952601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:37.959228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24e084-2172-441e-b2b3-1369df53ed1a None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:38.049317 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f278e37-3b1a-4523-abf0-a9727a68ee4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:38.050629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f278e37-3b1a-4523-abf0-a9727a68ee4b None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:38.051270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f278e37-3b1a-4523-abf0-a9727a68ee4b None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:38.051913 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f278e37-3b1a-4523-abf0-a9727a68ee4b None admin] PATH_INFO: `/v3/groups/59e391a4350244c9888947a0896ee4e2` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:38.055128 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f278e37-3b1a-4523-abf0-a9727a68ee4b None admin] RBAC: Authorizing `identity:get_group(group_id=59e391a4350244c9888947a0896ee4e2)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:38.058749 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f278e37-3b1a-4523-abf0-a9727a68ee4b None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:38.063354 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 190/370] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 7 13:08:37 2022] GET /identity/v3/groups/59e391a4350244c9888947a0896ee4e2 => generated 226 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:38.065189 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f278e37-3b1a-4523-abf0-a9727a68ee4b None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:38.089374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409cc628-b0a0-406e-8a76-0fc41cc62d6c None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:38.167840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841e162-6913-4cb3-bb9e-336b9953c283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:38.168937 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841e162-6913-4cb3-bb9e-336b9953c283 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:38.169439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841e162-6913-4cb3-bb9e-336b9953c283 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:38.169995 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841e162-6913-4cb3-bb9e-336b9953c283 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:38.171383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841e162-6913-4cb3-bb9e-336b9953c283 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:38.172505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841e162-6913-4cb3-bb9e-336b9953c283 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:38.175841 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d841e162-6913-4cb3-bb9e-336b9953c283 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:38.201925 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 181/371] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:38.201925 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841e162-6913-4cb3-bb9e-336b9953c283 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:38.259382 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:38.324262 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f278e37-3b1a-4523-abf0-a9727a68ee4b None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:38.417681 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada8df0-7ac2-40b2-a133-726889750bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:38.419815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada8df0-7ac2-40b2-a133-726889750bbf None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:38.420663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada8df0-7ac2-40b2-a133-726889750bbf None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:38.421911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada8df0-7ac2-40b2-a133-726889750bbf None admin] PATH_INFO: `/v3/users/da94718d76c2418689b7ad9beb7bc01e` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:38.425835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada8df0-7ac2-40b2-a133-726889750bbf None admin] RBAC: Authorizing `identity:get_user(user_id=da94718d76c2418689b7ad9beb7bc01e)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:38.429767 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada8df0-7ac2-40b2-a133-726889750bbf None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:38.434553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 191/372] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:08:38 2022] GET /identity/v3/users/da94718d76c2418689b7ad9beb7bc01e => generated 293 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:38.442953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841e162-6913-4cb3-bb9e-336b9953c283 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:38.445232 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada8df0-7ac2-40b2-a133-726889750bbf None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:38.532441 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce3e40-23a9-4224-adfc-181f2f196647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:38.534671 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce3e40-23a9-4224-adfc-181f2f196647 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:38.535325 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce3e40-23a9-4224-adfc-181f2f196647 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:38.535945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce3e40-23a9-4224-adfc-181f2f196647 None admin] PATH_INFO: `/v3/groups/59e391a4350244c9888947a0896ee4e2` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:38.538966 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce3e40-23a9-4224-adfc-181f2f196647 None admin] RBAC: Authorizing `identity:get_group(group_id=59e391a4350244c9888947a0896ee4e2)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:38.541630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce3e40-23a9-4224-adfc-181f2f196647 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:38.545438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 182/373] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 7 13:08:38 2022] GET /identity/v3/groups/59e391a4350244c9888947a0896ee4e2 => generated 226 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:38.546954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce3e40-23a9-4224-adfc-181f2f196647 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:38.548959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:38.704258 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada8df0-7ac2-40b2-a133-726889750bbf None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:38.771783 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:38.786832 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9de77-0496-4f8b-9729-b2b5e63b394c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:38.788617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9de77-0496-4f8b-9729-b2b5e63b394c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:38.789313 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9de77-0496-4f8b-9729-b2b5e63b394c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:38.790030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9de77-0496-4f8b-9729-b2b5e63b394c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:38.791757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9de77-0496-4f8b-9729-b2b5e63b394c None admin] RBAC: Adding query filter params (group.id=59e391a4350244c9888947a0896ee4e2, role.id=bcc52f73cf03442e922a4757908334de, scope.project.id=1fcdd13492724455ae65d6e80d2f41bc) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:38.792077 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce3e40-23a9-4224-adfc-181f2f196647 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:38.792751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9de77-0496-4f8b-9729-b2b5e63b394c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:38.796159 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9de77-0496-4f8b-9729-b2b5e63b394c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:38.809318 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 192/374] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 7 13:08:38 2022] GET /identity/v3/role_assignments?group.id=59e391a4350244c9888947a0896ee4e2&scope.project.id=1fcdd13492724455ae65d6e80d2f41bc&role.id=bcc52f73cf03442e922a4757908334de => generated 260 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:38.812382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9de77-0496-4f8b-9729-b2b5e63b394c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:38.906601 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ba379-4b1e-4d87-80ce-45c08b2f0980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:38.911578 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ba379-4b1e-4d87-80ce-45c08b2f0980 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:38.912391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ba379-4b1e-4d87-80ce-45c08b2f0980 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:38.913749 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ba379-4b1e-4d87-80ce-45c08b2f0980 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:39.091046 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ba379-4b1e-4d87-80ce-45c08b2f0980 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:39.091046 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ba379-4b1e-4d87-80ce-45c08b2f0980 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:39.193234 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:39.205869 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9de77-0496-4f8b-9729-b2b5e63b394c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:39.332220 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed96b5-4675-4790-b4a5-3d708f9eb79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:39.333382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed96b5-4675-4790-b4a5-3d708f9eb79c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:39.333938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed96b5-4675-4790-b4a5-3d708f9eb79c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:39.334412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed96b5-4675-4790-b4a5-3d708f9eb79c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:39.335884 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed96b5-4675-4790-b4a5-3d708f9eb79c None admin] RBAC: Adding query filter params (role.id=0503208661c44c4f8f52e30b4f21fff7, user.id=da94718d76c2418689b7ad9beb7bc01e, scope.system=all) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:39.336518 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed96b5-4675-4790-b4a5-3d708f9eb79c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:39.351129 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed96b5-4675-4790-b4a5-3d708f9eb79c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:39.374904 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:39.392249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 193/375] 10.222.0.62 () {56 vars in 1308 bytes} [Wed Sep 7 13:08:38 2022] GET /identity/v3/role_assignments?user.id=da94718d76c2418689b7ad9beb7bc01e&scope.system=all&role.id=0503208661c44c4f8f52e30b4f21fff7 => generated 510 bytes in 582 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:39.394181 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 183/376] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:08:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 848 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:08:39.396070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed96b5-4675-4790-b4a5-3d708f9eb79c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:39.406336 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ba379-4b1e-4d87-80ce-45c08b2f0980 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:39.607075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ba379-4b1e-4d87-80ce-45c08b2f0980 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:39.685933 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af07fd-8389-4bdf-baee-22f14aa45d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:39.687710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af07fd-8389-4bdf-baee-22f14aa45d51 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:39.688550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af07fd-8389-4bdf-baee-22f14aa45d51 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:39.689353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af07fd-8389-4bdf-baee-22f14aa45d51 None admin] PATH_INFO: `/v3/projects/0ee134892b8643809999afc672e86085` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:39.692553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af07fd-8389-4bdf-baee-22f14aa45d51 None admin] RBAC: Authorizing `identity:get_project(project_id=0ee134892b8643809999afc672e86085)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:39.695961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af07fd-8389-4bdf-baee-22f14aa45d51 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:39.700438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 184/377] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:39 2022] GET /identity/v3/projects/0ee134892b8643809999afc672e86085 => generated 344 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:08:39.713278 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af07fd-8389-4bdf-baee-22f14aa45d51 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:39.836462 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed96b5-4675-4790-b4a5-3d708f9eb79c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:39.887025 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:39.927212 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f232b-652e-4281-bd89-1e9c705d5f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:39.928212 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f232b-652e-4281-bd89-1e9c705d5f48 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:39.928669 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f232b-652e-4281-bd89-1e9c705d5f48 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:39.929117 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f232b-652e-4281-bd89-1e9c705d5f48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:39.930436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f232b-652e-4281-bd89-1e9c705d5f48 None admin] RBAC: Adding query filter params (group.id=59e391a4350244c9888947a0896ee4e2, role.id=fc0a7877dd0f4903a2a6fcc01a829c6a, scope.project.id=920dfa8ad35b4a429aea367ae3067b8e) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:39.930940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f232b-652e-4281-bd89-1e9c705d5f48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:39.933576 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f232b-652e-4281-bd89-1e9c705d5f48 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:39.944625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 194/378] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 7 13:08:39 2022] GET /identity/v3/role_assignments?group.id=59e391a4350244c9888947a0896ee4e2&scope.project.id=920dfa8ad35b4a429aea367ae3067b8e&role.id=fc0a7877dd0f4903a2a6fcc01a829c6a => generated 260 bytes in 551 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:39.947751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f00695-6a1a-49a5-9b4d-986926be9965 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:39.948200 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f00695-6a1a-49a5-9b4d-986926be9965 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:39.948647 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f00695-6a1a-49a5-9b4d-986926be9965 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:39.951599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 195/379] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:39.971251 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9873f6-132c-471d-b0d5-1d76dbedf9f3 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:39.971700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9873f6-132c-471d-b0d5-1d76dbedf9f3 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:39.972120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9873f6-132c-471d-b0d5-1d76dbedf9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:39.996614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af07fd-8389-4bdf-baee-22f14aa45d51 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:40.033810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-be9873f6-132c-471d-b0d5-1d76dbedf9f3 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:40.036847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-be9873f6-132c-471d-b0d5-1d76dbedf9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:40.071974 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d05709-aab9-4dae-bf43-11ee6bd74dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:40.073682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d05709-aab9-4dae-bf43-11ee6bd74dda None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:40.074572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d05709-aab9-4dae-bf43-11ee6bd74dda None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:40.075457 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d05709-aab9-4dae-bf43-11ee6bd74dda None admin] PATH_INFO: `/v3/groups/ef70b838a8bf4e6e871179b8b339e868` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:40.086942 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d05709-aab9-4dae-bf43-11ee6bd74dda None admin] RBAC: Authorizing `identity:get_group(group_id=ef70b838a8bf4e6e871179b8b339e868)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:40.090338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d05709-aab9-4dae-bf43-11ee6bd74dda None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:40.094290 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 185/380] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 7 13:08:39 2022] GET /identity/v3/groups/ef70b838a8bf4e6e871179b8b339e868 => generated 229 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:40.096614 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:40.105181 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d05709-aab9-4dae-bf43-11ee6bd74dda None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:40.318290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 196/381] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:08:40.383259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d05709-aab9-4dae-bf43-11ee6bd74dda None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:40.445710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:40.485802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475818bf-d927-4665-a23a-129cd0094bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:40.487245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475818bf-d927-4665-a23a-129cd0094bba None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:40.487738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475818bf-d927-4665-a23a-129cd0094bba None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:40.488222 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475818bf-d927-4665-a23a-129cd0094bba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:40.489822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475818bf-d927-4665-a23a-129cd0094bba None admin] RBAC: Adding query filter params (group.id=ef70b838a8bf4e6e871179b8b339e868, role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=0ee134892b8643809999afc672e86085) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:40.490626 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475818bf-d927-4665-a23a-129cd0094bba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:40.493434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475818bf-d927-4665-a23a-129cd0094bba None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:40.512554 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 186/382] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 7 13:08:40 2022] GET /identity/v3/role_assignments?group.id=ef70b838a8bf4e6e871179b8b339e868&scope.project.id=0ee134892b8643809999afc672e86085&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 260 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:08:40.937730 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9873f6-132c-471d-b0d5-1d76dbedf9f3 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:41.192699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9873f6-132c-471d-b0d5-1d76dbedf9f3 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:41.273450 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5149aa-ae1c-43b4-9c80-613539ee1b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:41.274631 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5149aa-ae1c-43b4-9c80-613539ee1b47 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:41.275181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5149aa-ae1c-43b4-9c80-613539ee1b47 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:41.275683 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5149aa-ae1c-43b4-9c80-613539ee1b47 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:41.302730 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5149aa-ae1c-43b4-9c80-613539ee1b47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:41.317455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-2c5149aa-ae1c-43b4-9c80-613539ee1b47 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:08:41.317455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:41.317455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:41.317455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:41.317455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:41.317455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:41.317455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:41.317455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:41.317455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:41.317455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:41.317455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:41.317455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:41.317455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:41.317455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:41.317455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:41.317455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:41.319050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:41.319050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:41.319050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:41.319050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:41.319050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:41.319050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:41.319050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:41.319050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:41.319050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:41.319050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:41.319050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:41.319050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:41.319050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:41.319050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:41.319050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:41.319050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:41.321552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:41.321552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:41.321552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:41.321552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:41.321552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:41.321552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:41.321552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:41.321552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:41.321552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:41.321552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:41.321552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:08:41.321552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:08:41.321552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 197/383] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:08:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:41.331690 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475818bf-d927-4665-a23a-129cd0094bba None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:41.667499 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475818bf-d927-4665-a23a-129cd0094bba None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:41.761113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb95887-76b5-46cb-9c70-c68b9bbf625d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:41.762730 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb95887-76b5-46cb-9c70-c68b9bbf625d None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:41.763598 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb95887-76b5-46cb-9c70-c68b9bbf625d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:41.764371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb95887-76b5-46cb-9c70-c68b9bbf625d None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:41.765967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb95887-76b5-46cb-9c70-c68b9bbf625d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:41.766940 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb95887-76b5-46cb-9c70-c68b9bbf625d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:41.770163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb95887-76b5-46cb-9c70-c68b9bbf625d None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:41.795425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 187/384] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:08:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 466 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:41.805578 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5149aa-ae1c-43b4-9c80-613539ee1b47 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:41.867238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:42.031781 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b0a6f-8e29-412b-8cf3-b7e0ef0a5554 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:42.033783 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b0a6f-8e29-412b-8cf3-b7e0ef0a5554 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:42.034724 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b0a6f-8e29-412b-8cf3-b7e0ef0a5554 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:42.038516 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 188/385] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:42.048334 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd2607-2e2b-412c-9baa-dd388e15291c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:42.049472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd2607-2e2b-412c-9baa-dd388e15291c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:42.050351 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd2607-2e2b-412c-9baa-dd388e15291c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:42.125589 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-52fd2607-2e2b-412c-9baa-dd388e15291c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:42.127912 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-52fd2607-2e2b-412c-9baa-dd388e15291c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:42.142184 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5149aa-ae1c-43b4-9c80-613539ee1b47 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:42.226340 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e80a5-1f73-48b1-973a-8872e0a1469e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:42.227476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e80a5-1f73-48b1-973a-8872e0a1469e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:42.227949 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e80a5-1f73-48b1-973a-8872e0a1469e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:42.228415 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e80a5-1f73-48b1-973a-8872e0a1469e None admin] PATH_INFO: `/v3/groups/59e391a4350244c9888947a0896ee4e2` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:42.230963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e80a5-1f73-48b1-973a-8872e0a1469e None admin] RBAC: Authorizing `identity:get_group(group_id=59e391a4350244c9888947a0896ee4e2)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:42.239013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e80a5-1f73-48b1-973a-8872e0a1469e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:42.239013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 198/386] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 7 13:08:41 2022] GET /identity/v3/groups/59e391a4350244c9888947a0896ee4e2 => generated 226 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:42.248828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e80a5-1f73-48b1-973a-8872e0a1469e None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:42.315772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:42.369990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 189/387] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:08:42.451680 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:42.512812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e80a5-1f73-48b1-973a-8872e0a1469e None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:42.624921 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec683c9-da5c-461e-8961-513cde8b50af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:42.625940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec683c9-da5c-461e-8961-513cde8b50af None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:42.626431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec683c9-da5c-461e-8961-513cde8b50af None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:42.627012 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec683c9-da5c-461e-8961-513cde8b50af None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:42.629253 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec683c9-da5c-461e-8961-513cde8b50af None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:42.631928 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec683c9-da5c-461e-8961-513cde8b50af None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:42.635039 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 199/388] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:42 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:42.645282 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd2607-2e2b-412c-9baa-dd388e15291c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:42.895209 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd2607-2e2b-412c-9baa-dd388e15291c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:42.980419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881ab3f-fb28-4042-aa91-aeb5f22ea007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:42.981632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881ab3f-fb28-4042-aa91-aeb5f22ea007 None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:42.982138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881ab3f-fb28-4042-aa91-aeb5f22ea007 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:42.982642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881ab3f-fb28-4042-aa91-aeb5f22ea007 None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc/groups/59e391a4350244c9888947a0896ee4e2/roles/fc0a7877dd0f4903a2a6fcc01a829c6a` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:42.987834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881ab3f-fb28-4042-aa91-aeb5f22ea007 None admin] RBAC: Authorizing `identity:create_grant(project_id=1fcdd13492724455ae65d6e80d2f41bc, group_id=59e391a4350244c9888947a0896ee4e2, role_id=fc0a7877dd0f4903a2a6fcc01a829c6a)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:42.990770 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881ab3f-fb28-4042-aa91-aeb5f22ea007 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:43.063697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 190/389] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Sep 7 13:08:42 2022] PUT /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc/groups/59e391a4350244c9888947a0896ee4e2/roles/fc0a7877dd0f4903a2a6fcc01a829c6a => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:43.170035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec683c9-da5c-461e-8961-513cde8b50af None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:43.448810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec683c9-da5c-461e-8961-513cde8b50af None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:43.550911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557c97e-fd87-4712-b12d-d10fa232ab34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:43.552098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557c97e-fd87-4712-b12d-d10fa232ab34 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:43.552600 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557c97e-fd87-4712-b12d-d10fa232ab34 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:43.553098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557c97e-fd87-4712-b12d-d10fa232ab34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:43.576518 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557c97e-fd87-4712-b12d-d10fa232ab34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:43.587049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-1557c97e-fd87-4712-b12d-d10fa232ab34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:43.587049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:43.587049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:43.587049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:43.587049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:43.587049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:43.587049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:43.587049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:43.587049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:43.587049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:43.587049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:43.587049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:43.587049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:43.587049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:43.587049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:43.587049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:43.588834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:43.588834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:43.588834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:43.588834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:43.588834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:43.588834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:43.588834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:43.588834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:43.588834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:43.588834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:43.588834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:43.588834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:43.588834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:43.588834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:43.588834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:43.588834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:43.590476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:43.590476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:43.590476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:43.590476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:43.590476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:43.590476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:43.590476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:43.590476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:43.590476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:43.590476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:43.590476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:43.590476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:08:43.590476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 200/390] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:43.600454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881ab3f-fb28-4042-aa91-aeb5f22ea007 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:43.696780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:43.840340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881ab3f-fb28-4042-aa91-aeb5f22ea007 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:43.921317 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dacf9-6409-45b5-91b3-c260f526ecb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:43.922381 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dacf9-6409-45b5-91b3-c260f526ecb0 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:43.922887 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dacf9-6409-45b5-91b3-c260f526ecb0 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:43.923409 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dacf9-6409-45b5-91b3-c260f526ecb0 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:43.926365 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7dacf9-6409-45b5-91b3-c260f526ecb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:43.926932 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7dacf9-6409-45b5-91b3-c260f526ecb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:43.929682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7dacf9-6409-45b5-91b3-c260f526ecb0 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:43.948610 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 191/391] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:43.960216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557c97e-fd87-4712-b12d-d10fa232ab34 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:43.996513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:44.021861 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0b744-a4ba-4226-a6cc-b14f2332fab2 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:44.031134 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0b744-a4ba-4226-a6cc-b14f2332fab2 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:44.031134 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0b744-a4ba-4226-a6cc-b14f2332fab2 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:44.031134 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 192/392] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:08:44.038701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fdd80-515b-49de-86c2-f2a2ca175c12 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:44.039734 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fdd80-515b-49de-86c2-f2a2ca175c12 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:44.040554 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fdd80-515b-49de-86c2-f2a2ca175c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:44.080533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-a02fdd80-515b-49de-86c2-f2a2ca175c12 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:44.084855 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a02fdd80-515b-49de-86c2-f2a2ca175c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:44.272726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557c97e-fd87-4712-b12d-d10fa232ab34 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:44.348825 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 193/393] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:08:44.421901 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe63ca7-f9e6-44df-944c-0b0ec7b358c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:44.422604 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe63ca7-f9e6-44df-944c-0b0ec7b358c3 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:44.422885 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe63ca7-f9e6-44df-944c-0b0ec7b358c3 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:44.423189 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe63ca7-f9e6-44df-944c-0b0ec7b358c3 None admin] PATH_INFO: `/v3/groups/59e391a4350244c9888947a0896ee4e2` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:44.428031 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe63ca7-f9e6-44df-944c-0b0ec7b358c3 None admin] RBAC: Authorizing `identity:get_group(group_id=59e391a4350244c9888947a0896ee4e2)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:44.429623 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe63ca7-f9e6-44df-944c-0b0ec7b358c3 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:44.431789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 201/394] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 7 13:08:43 2022] GET /identity/v3/groups/59e391a4350244c9888947a0896ee4e2 => generated 226 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:44.441916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02fdd80-515b-49de-86c2-f2a2ca175c12 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:44.515870 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:44.712254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02fdd80-515b-49de-86c2-f2a2ca175c12 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:44.785072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e5349-095d-48a4-a117-cd7d59f1cc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:44.786128 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e5349-095d-48a4-a117-cd7d59f1cc3b None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:44.786609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e5349-095d-48a4-a117-cd7d59f1cc3b None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:44.787103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e5349-095d-48a4-a117-cd7d59f1cc3b None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:44.788908 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e5349-095d-48a4-a117-cd7d59f1cc3b None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:44.790842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e5349-095d-48a4-a117-cd7d59f1cc3b None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:44.792991 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 194/395] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:44 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:44.804157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe63ca7-f9e6-44df-944c-0b0ec7b358c3 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:44.987521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e5349-095d-48a4-a117-cd7d59f1cc3b None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:45.008329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:45.080900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe63ca7-f9e6-44df-944c-0b0ec7b358c3 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:45.197722 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a545a14-13b5-4bfb-a0b7-43addd24886f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:45.200583 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a545a14-13b5-4bfb-a0b7-43addd24886f None admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:45.201581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a545a14-13b5-4bfb-a0b7-43addd24886f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:45.202359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a545a14-13b5-4bfb-a0b7-43addd24886f None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e/groups/59e391a4350244c9888947a0896ee4e2/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:45.207976 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a545a14-13b5-4bfb-a0b7-43addd24886f None admin] RBAC: Authorizing `identity:create_grant(project_id=920dfa8ad35b4a429aea367ae3067b8e, group_id=59e391a4350244c9888947a0896ee4e2, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:45.212446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a545a14-13b5-4bfb-a0b7-43addd24886f None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:45.310689 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:45.317788 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e5349-095d-48a4-a117-cd7d59f1cc3b None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:45.387695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 202/396] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Sep 7 13:08:44 2022] PUT /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e/groups/59e391a4350244c9888947a0896ee4e2/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:45.402695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e7424-2f5c-4a5c-a9dc-d067aad93598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:45.403673 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e7424-2f5c-4a5c-a9dc-d067aad93598 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:45.404144 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e7424-2f5c-4a5c-a9dc-d067aad93598 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:45.404616 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e7424-2f5c-4a5c-a9dc-d067aad93598 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:45.410016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725b655-57a1-468f-93c7-1eb64e8c9a06 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:45.419620 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725b655-57a1-468f-93c7-1eb64e8c9a06 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:45.419620 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725b655-57a1-468f-93c7-1eb64e8c9a06 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:45.419620 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 203/397] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:08:45.427971 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7504be-c876-4d80-98e6-dd1d76c9bd6c None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:45.428449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7504be-c876-4d80-98e6-dd1d76c9bd6c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:45.428945 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7504be-c876-4d80-98e6-dd1d76c9bd6c None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:45.440350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 204/398] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:45 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:45.440604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e7424-2f5c-4a5c-a9dc-d067aad93598 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:45.442857 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79123e6-1ac8-420f-b35f-42c8d1d8e8c0 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:45.443379 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79123e6-1ac8-420f-b35f-42c8d1d8e8c0 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:45.443841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79123e6-1ac8-420f-b35f-42c8d1d8e8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:45.451251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-271e7424-2f5c-4a5c-a9dc-d067aad93598 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:45.451251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:45.451251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:45.451251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:45.451251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:45.451251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:45.451251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:45.451251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:45.451251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:45.451251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:45.451251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:45.451251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:45.451251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:45.451251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:45.451251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:45.451251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:45.454865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:45.454865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:45.454865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:45.454865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:45.454865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:45.454865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:45.454865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:45.454865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:45.454865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:45.454865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:45.454865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:45.454865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:45.454865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:45.454865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:45.454865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:45.454865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:45.458839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:45.458839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:45.458839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:45.458839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:45.458839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:45.458839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:45.458839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:45.458839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:45.458839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:45.458839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:45.458839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:45.458839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:45.458839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 195/399] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 468 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:45.458839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4deb61-7f6d-4082-9836-bdd5d3f832cb None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:45.467066 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4deb61-7f6d-4082-9836-bdd5d3f832cb None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:45.467066 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4deb61-7f6d-4082-9836-bdd5d3f832cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:45.495111 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-d79123e6-1ac8-420f-b35f-42c8d1d8e8c0 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:45.497678 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d79123e6-1ac8-420f-b35f-42c8d1d8e8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:45.500170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-5b4deb61-7f6d-4082-9836-bdd5d3f832cb None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:45.503858 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5b4deb61-7f6d-4082-9836-bdd5d3f832cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:45.633315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 196/400] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:45.639241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4deb61-7f6d-4082-9836-bdd5d3f832cb None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:45.643974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 205/401] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:45.819863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:45.838800 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4deb61-7f6d-4082-9836-bdd5d3f832cb None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:45.928127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08baab8a-6a63-4fa3-81b5-c3d58a42549d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:45.929080 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08baab8a-6a63-4fa3-81b5-c3d58a42549d None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:45.929562 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08baab8a-6a63-4fa3-81b5-c3d58a42549d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:45.930049 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08baab8a-6a63-4fa3-81b5-c3d58a42549d None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:45.931348 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08baab8a-6a63-4fa3-81b5-c3d58a42549d None admin] RBAC: Adding query filter params (name=member) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:45.931898 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08baab8a-6a63-4fa3-81b5-c3d58a42549d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:45.934544 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08baab8a-6a63-4fa3-81b5-c3d58a42549d None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:45.956727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 197/402] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:45.968756 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79123e6-1ac8-420f-b35f-42c8d1d8e8c0 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:46.019113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08baab8a-6a63-4fa3-81b5-c3d58a42549d None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:46.144291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:46.197084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08baab8a-6a63-4fa3-81b5-c3d58a42549d None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:46.225039 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79123e6-1ac8-420f-b35f-42c8d1d8e8c0 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:46.246925 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b757d-b870-4f9f-a34e-cf6aacdc7fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:46.247787 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b757d-b870-4f9f-a34e-cf6aacdc7fd0 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:46.248299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b757d-b870-4f9f-a34e-cf6aacdc7fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:46.248789 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b757d-b870-4f9f-a34e-cf6aacdc7fd0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:46.270163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b757d-b870-4f9f-a34e-cf6aacdc7fd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:46.280988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-184b757d-b870-4f9f-a34e-cf6aacdc7fd0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:08:46.280988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:46.280988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:46.280988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:46.280988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:46.280988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:46.280988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:46.280988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:46.280988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:46.280988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:46.280988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:46.280988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:46.280988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:46.280988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:46.280988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:46.280988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:46.282696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:46.282696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:46.282696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:46.282696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:46.282696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:46.282696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:46.282696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:46.282696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:46.282696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:46.282696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:46.282696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:46.282696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:46.282696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:46.282696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:46.282696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:46.282696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:46.286206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:46.286206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:46.286206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:46.286206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:46.286206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:46.286206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:46.286206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:46.286206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:46.286206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:46.286206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:46.286206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:08:46.286206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:46.286206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 198/403] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:08:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:46.286206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b757d-b870-4f9f-a34e-cf6aacdc7fd0 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:46.304987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:46.311532 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbeb48d-e51e-4a52-9817-826692f10d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:46.313044 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbeb48d-e51e-4a52-9817-826692f10d4c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:46.313854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbeb48d-e51e-4a52-9817-826692f10d4c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:46.316226 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbeb48d-e51e-4a52-9817-826692f10d4c None admin] PATH_INFO: `/v3/groups/59e391a4350244c9888947a0896ee4e2` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:46.318727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbeb48d-e51e-4a52-9817-826692f10d4c None admin] RBAC: Authorizing `identity:get_group(group_id=59e391a4350244c9888947a0896ee4e2)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:46.321721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbeb48d-e51e-4a52-9817-826692f10d4c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:46.325492 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 206/404] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 7 13:08:45 2022] GET /identity/v3/groups/59e391a4350244c9888947a0896ee4e2 => generated 226 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:46.333614 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbeb48d-e51e-4a52-9817-826692f10d4c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:46.422384 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b757d-b870-4f9f-a34e-cf6aacdc7fd0 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:46.465918 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619eeef-edd8-4e78-9a46-f61855e58def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:46.466477 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619eeef-edd8-4e78-9a46-f61855e58def None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:46.466773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619eeef-edd8-4e78-9a46-f61855e58def None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:46.467094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619eeef-edd8-4e78-9a46-f61855e58def None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:46.483082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3619eeef-edd8-4e78-9a46-f61855e58def None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:46.493566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-3619eeef-edd8-4e78-9a46-f61855e58def None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:08:46.493566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:46.493566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:46.493566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:46.493566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:46.493566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:46.493566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:46.493566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:46.493566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:46.493566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:46.493566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:46.493566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:46.493566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:46.493566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:46.493566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:46.493566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:46.495264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:46.495264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:46.495264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:46.495264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:46.495264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:46.495264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:46.495264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:46.495264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:46.495264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:46.495264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:46.495264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:46.495264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:46.495264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:46.495264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:46.495264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:46.495264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:46.496659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:46.496659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:46.496659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:46.496659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:46.496659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:46.496659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:46.496659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:46.496659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:46.496659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:46.496659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:46.496659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:08:46.496659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:46.503469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 199/405] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:08:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:46.504428 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619eeef-edd8-4e78-9a46-f61855e58def None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:46.568355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbeb48d-e51e-4a52-9817-826692f10d4c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:46.618397 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:46.642024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0e133-ea39-4f3b-bbbd-48670194f1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:46.644738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0e133-ea39-4f3b-bbbd-48670194f1ce None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:46.645519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0e133-ea39-4f3b-bbbd-48670194f1ce None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:46.646286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0e133-ea39-4f3b-bbbd-48670194f1ce None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:46.647751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0e133-ea39-4f3b-bbbd-48670194f1ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:46.648520 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0e133-ea39-4f3b-bbbd-48670194f1ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:46.651027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0e133-ea39-4f3b-bbbd-48670194f1ce None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:46.653858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:46.662051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619eeef-edd8-4e78-9a46-f61855e58def None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:46.671389 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 207/406] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:46.673263 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0e133-ea39-4f3b-bbbd-48670194f1ce None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:46.709017 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b995027-1287-4a6e-970d-95169f1cfca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:46.709527 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b995027-1287-4a6e-970d-95169f1cfca2 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:46.709893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b995027-1287-4a6e-970d-95169f1cfca2 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:46.710313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b995027-1287-4a6e-970d-95169f1cfca2 None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:46.711897 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b995027-1287-4a6e-970d-95169f1cfca2 None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:46.713481 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b995027-1287-4a6e-970d-95169f1cfca2 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:46.721973 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 200/407] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:46 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:46.721973 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b995027-1287-4a6e-970d-95169f1cfca2 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:46.878228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0e133-ea39-4f3b-bbbd-48670194f1ce None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:46.905930 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b995027-1287-4a6e-970d-95169f1cfca2 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:46.929840 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958967f9-4bc0-479e-a4b3-e506a8598357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:46.931470 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958967f9-4bc0-479e-a4b3-e506a8598357 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:46.932324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958967f9-4bc0-479e-a4b3-e506a8598357 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:46.933029 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958967f9-4bc0-479e-a4b3-e506a8598357 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:46.934456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958967f9-4bc0-479e-a4b3-e506a8598357 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:46.935280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958967f9-4bc0-479e-a4b3-e506a8598357 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:46.938022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958967f9-4bc0-479e-a4b3-e506a8598357 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:46.956797 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 208/408] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:08:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:46.958868 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958967f9-4bc0-479e-a4b3-e506a8598357 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:46.987072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5a083-2d99-4371-84eb-49e88840013b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:46.987705 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5a083-2d99-4371-84eb-49e88840013b None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:46.988217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5a083-2d99-4371-84eb-49e88840013b None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:46.988755 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5a083-2d99-4371-84eb-49e88840013b None admin] PATH_INFO: `/v3/groups/ef70b838a8bf4e6e871179b8b339e868` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:46.991066 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5a083-2d99-4371-84eb-49e88840013b None admin] RBAC: Authorizing `identity:get_group(group_id=ef70b838a8bf4e6e871179b8b339e868)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:46.993768 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5a083-2d99-4371-84eb-49e88840013b None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:46.997323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 201/409] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 7 13:08:46 2022] GET /identity/v3/groups/ef70b838a8bf4e6e871179b8b339e868 => generated 229 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:46.999006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5a083-2d99-4371-84eb-49e88840013b None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:47.021629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:47.087551 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:47.182648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958967f9-4bc0-479e-a4b3-e506a8598357 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:47.208817 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5a083-2d99-4371-84eb-49e88840013b None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:47.253704 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e82222-c86a-4daf-bd8e-b59c071c6c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:47.254685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e82222-c86a-4daf-bd8e-b59c071c6c29 None admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:47.255087 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e82222-c86a-4daf-bd8e-b59c071c6c29 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:47.255432 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e82222-c86a-4daf-bd8e-b59c071c6c29 None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc/groups/59e391a4350244c9888947a0896ee4e2/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:47.258472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e82222-c86a-4daf-bd8e-b59c071c6c29 None admin] RBAC: Authorizing `identity:create_grant(project_id=1fcdd13492724455ae65d6e80d2f41bc, group_id=59e391a4350244c9888947a0896ee4e2, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:47.260993 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e82222-c86a-4daf-bd8e-b59c071c6c29 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:47.284175 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6619627-c12e-4e3b-9b1c-e05bde911268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:47.285268 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6619627-c12e-4e3b-9b1c-e05bde911268 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:47.285746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6619627-c12e-4e3b-9b1c-e05bde911268 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:47.286223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6619627-c12e-4e3b-9b1c-e05bde911268 None admin] PATH_INFO: `/v3/groups/59e391a4350244c9888947a0896ee4e2` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:47.288526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6619627-c12e-4e3b-9b1c-e05bde911268 None admin] RBAC: Authorizing `identity:get_group(group_id=59e391a4350244c9888947a0896ee4e2)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:47.291389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6619627-c12e-4e3b-9b1c-e05bde911268 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:47.294490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 202/410] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 7 13:08:46 2022] GET /identity/v3/groups/59e391a4350244c9888947a0896ee4e2 => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:47.295979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6619627-c12e-4e3b-9b1c-e05bde911268 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:47.375698 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 209/411] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Sep 7 13:08:46 2022] PUT /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc/groups/59e391a4350244c9888947a0896ee4e2/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:47.380646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9271e1-ec90-4780-ba2b-c00db31181e0 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:47.381409 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9271e1-ec90-4780-ba2b-c00db31181e0 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:47.381853 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9271e1-ec90-4780-ba2b-c00db31181e0 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:47.384208 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 210/412] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:47.385907 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9271e1-ec90-4780-ba2b-c00db31181e0 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:47.472335 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:47.520977 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6619627-c12e-4e3b-9b1c-e05bde911268 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:47.579366 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a440f0-677f-4948-a856-f0911f96e23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:47.581469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a440f0-677f-4948-a856-f0911f96e23a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:47.582444 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a440f0-677f-4948-a856-f0911f96e23a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:47.583178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a440f0-677f-4948-a856-f0911f96e23a None admin] PATH_INFO: `/v3/projects/0ee134892b8643809999afc672e86085` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:47.585937 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a440f0-677f-4948-a856-f0911f96e23a None admin] RBAC: Authorizing `identity:get_project(project_id=0ee134892b8643809999afc672e86085)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:47.588456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a440f0-677f-4948-a856-f0911f96e23a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:47.591454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 203/413] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:47 2022] GET /identity/v3/projects/0ee134892b8643809999afc672e86085 => generated 344 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:47.593285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b73b3-3d9c-4501-b71d-580e2fc4830c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:47.593883 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b73b3-3d9c-4501-b71d-580e2fc4830c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:47.594417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b73b3-3d9c-4501-b71d-580e2fc4830c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:47.618829 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-580b73b3-3d9c-4501-b71d-580e2fc4830c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:47.620677 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-580b73b3-3d9c-4501-b71d-580e2fc4830c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:47.640610 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9271e1-ec90-4780-ba2b-c00db31181e0 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:47.676256 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:47.703030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa44d6-d9f2-4b39-ae8e-70928b338a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:47.704523 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa44d6-d9f2-4b39-ae8e-70928b338a93 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:47.705342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa44d6-d9f2-4b39-ae8e-70928b338a93 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:47.706110 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa44d6-d9f2-4b39-ae8e-70928b338a93 None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:47.708859 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa44d6-d9f2-4b39-ae8e-70928b338a93 None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:47.711597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa44d6-d9f2-4b39-ae8e-70928b338a93 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:47.715549 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 211/414] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:47 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:47.719491 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa44d6-d9f2-4b39-ae8e-70928b338a93 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:47.741467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 204/415] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:47.742678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b73b3-3d9c-4501-b71d-580e2fc4830c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:47.864748 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:47.901295 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa44d6-d9f2-4b39-ae8e-70928b338a93 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:47.913864 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b73b3-3d9c-4501-b71d-580e2fc4830c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:47.967683 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bf92e-f738-42a6-8fed-36647105fb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:47.968728 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bf92e-f738-42a6-8fed-36647105fb11 None admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:47.969159 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bf92e-f738-42a6-8fed-36647105fb11 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:47.969596 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bf92e-f738-42a6-8fed-36647105fb11 None admin] PATH_INFO: `/v3/projects/0ee134892b8643809999afc672e86085/groups/ef70b838a8bf4e6e871179b8b339e868/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:47.973937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bf92e-f738-42a6-8fed-36647105fb11 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ee134892b8643809999afc672e86085, group_id=ef70b838a8bf4e6e871179b8b339e868, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:47.976721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bf92e-f738-42a6-8fed-36647105fb11 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:47.992179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0d668-d302-4855-8f75-f7d83640fffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:47.993630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0d668-d302-4855-8f75-f7d83640fffe None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:47.994084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0d668-d302-4855-8f75-f7d83640fffe None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:47.994522 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0d668-d302-4855-8f75-f7d83640fffe None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e/groups/59e391a4350244c9888947a0896ee4e2/roles/fc0a7877dd0f4903a2a6fcc01a829c6a` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:47.998768 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0d668-d302-4855-8f75-f7d83640fffe None admin] RBAC: Authorizing `identity:create_grant(project_id=920dfa8ad35b4a429aea367ae3067b8e, group_id=59e391a4350244c9888947a0896ee4e2, role_id=fc0a7877dd0f4903a2a6fcc01a829c6a)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:48.001696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0d668-d302-4855-8f75-f7d83640fffe None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:48.086823 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 212/416] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Sep 7 13:08:47 2022] PUT /identity/v3/projects/0ee134892b8643809999afc672e86085/groups/ef70b838a8bf4e6e871179b8b339e868/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:48.108585 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 205/417] 10.222.0.62 () {58 vars in 1442 bytes} [Wed Sep 7 13:08:47 2022] PUT /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e/groups/59e391a4350244c9888947a0896ee4e2/roles/fc0a7877dd0f4903a2a6fcc01a829c6a => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:08:48.442773 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bf92e-f738-42a6-8fed-36647105fb11 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:48.535086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:48.634222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bf92e-f738-42a6-8fed-36647105fb11 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:48.718958 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806a9a8-338b-4541-8e97-858d32186612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:48.722821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806a9a8-338b-4541-8e97-858d32186612 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:48.723715 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806a9a8-338b-4541-8e97-858d32186612 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:48.724462 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806a9a8-338b-4541-8e97-858d32186612 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:48.754818 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806a9a8-338b-4541-8e97-858d32186612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:48.768820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-5806a9a8-338b-4541-8e97-858d32186612 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:08:48.768820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:48.768820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:48.768820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:48.768820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:48.768820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:48.768820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:48.768820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:48.768820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:48.768820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:48.768820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:48.768820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:48.768820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:48.768820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:48.768820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:48.768820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:48.770584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:48.770584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:48.770584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:48.770584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:48.770584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:48.770584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:48.770584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:48.770584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:48.770584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:48.770584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:48.770584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:48.770584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:48.770584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:48.770584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:48.770584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:48.770584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:48.772177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:48.772177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:48.772177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:48.772177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:48.772177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:48.772177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:48.772177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:48.772177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:48.772177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:48.772177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:48.772177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:08:48.772177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:08:48.777109 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 213/418] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:08:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:48.785249 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0d668-d302-4855-8f75-f7d83640fffe None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:48.952619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c466d-284d-4ed4-9d8e-9a21ac89efa1 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:48.953578 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c466d-284d-4ed4-9d8e-9a21ac89efa1 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:48.954294 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c466d-284d-4ed4-9d8e-9a21ac89efa1 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:48.963339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 214/419] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:48.967464 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae86466-e6b2-452b-a6d0-a993e1fc78a8 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:48.968315 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae86466-e6b2-452b-a6d0-a993e1fc78a8 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:48.969102 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae86466-e6b2-452b-a6d0-a993e1fc78a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:49.024845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0d668-d302-4855-8f75-f7d83640fffe None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:49.034711 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-5ae86466-e6b2-452b-a6d0-a993e1fc78a8 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:49.037376 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-5ae86466-e6b2-452b-a6d0-a993e1fc78a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:49.102228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ffaf7-5168-49f9-ba09-1bf961bce4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:49.103256 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ffaf7-5168-49f9-ba09-1bf961bce4a3 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:49.103736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ffaf7-5168-49f9-ba09-1bf961bce4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:49.104200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ffaf7-5168-49f9-ba09-1bf961bce4a3 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:49.105420 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ffaf7-5168-49f9-ba09-1bf961bce4a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:49.105923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ffaf7-5168-49f9-ba09-1bf961bce4a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:49.108547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ffaf7-5168-49f9-ba09-1bf961bce4a3 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:49.111791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:49.136167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 206/420] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:08:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:49.143729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ffaf7-5168-49f9-ba09-1bf961bce4a3 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:49.238138 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 215/421] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:49.409079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ffaf7-5168-49f9-ba09-1bf961bce4a3 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:49.501181 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945c941-4f46-48cb-b00c-89622d012553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:49.502211 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945c941-4f46-48cb-b00c-89622d012553 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:49.502695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945c941-4f46-48cb-b00c-89622d012553 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:49.503207 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f945c941-4f46-48cb-b00c-89622d012553 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:49.685480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:49.701876 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f945c941-4f46-48cb-b00c-89622d012553 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:49.705161 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f945c941-4f46-48cb-b00c-89622d012553 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:49.981055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 207/422] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:08:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 839 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:08:49.990226 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae86466-e6b2-452b-a6d0-a993e1fc78a8 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:50.006852 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945c941-4f46-48cb-b00c-89622d012553 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:50.099954 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:50.313638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:50.338807 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f945c941-4f46-48cb-b00c-89622d012553 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:50.341030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae86466-e6b2-452b-a6d0-a993e1fc78a8 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:50.458886 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5950ea-1a55-4c32-a01e-a0247123e118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:50.460001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5950ea-1a55-4c32-a01e-a0247123e118 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:50.460483 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5950ea-1a55-4c32-a01e-a0247123e118 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:50.460951 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5950ea-1a55-4c32-a01e-a0247123e118 None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:50.464264 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5950ea-1a55-4c32-a01e-a0247123e118 None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:50.467218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5950ea-1a55-4c32-a01e-a0247123e118 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:50.472058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 216/423] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:49 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 483 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:50.473911 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d214b02-83a7-40d6-af26-ea892b1d575f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:50.475023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d214b02-83a7-40d6-af26-ea892b1d575f None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:50.475517 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d214b02-83a7-40d6-af26-ea892b1d575f None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:50.479796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d214b02-83a7-40d6-af26-ea892b1d575f None admin] PATH_INFO: `/v3/roles/member` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:50.486583 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5950ea-1a55-4c32-a01e-a0247123e118 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:50.519527 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d214b02-83a7-40d6-af26-ea892b1d575f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:50.540556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-3d214b02-83a7-40d6-af26-ea892b1d575f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:50.540556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:50.540556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:50.540556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:50.540556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:50.540556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:50.540556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:50.540556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:50.540556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:50.540556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:50.540556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:50.540556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:50.540556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:50.540556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:50.540556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:50.540556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:50.542312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:50.542312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:50.542312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:50.542312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:50.542312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:50.542312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:50.542312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:50.542312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:50.542312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:50.542312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:50.542312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:50.542312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:50.542312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:50.542312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:50.542312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:50.542312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:50.543944 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:50.543944 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:50.543944 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:50.543944 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:50.543944 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:50.543944 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:50.543944 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:50.543944 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:50.543944 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:50.543944 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:50.543944 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:50.543944 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:50.543944 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 208/424] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 537 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:08:50.559023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d214b02-83a7-40d6-af26-ea892b1d575f None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:50.849580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5950ea-1a55-4c32-a01e-a0247123e118 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:50.854671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:50.873331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d214b02-83a7-40d6-af26-ea892b1d575f None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:50.922217 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1c09b-6fde-4d39-b4c7-cdfba3588f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:50.924200 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1c09b-6fde-4d39-b4c7-cdfba3588f41 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:50.924723 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1c09b-6fde-4d39-b4c7-cdfba3588f41 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:50.925346 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1c09b-6fde-4d39-b4c7-cdfba3588f41 None admin] PATH_INFO: `/v3/groups/59e391a4350244c9888947a0896ee4e2` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:50.927598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1c09b-6fde-4d39-b4c7-cdfba3588f41 None admin] RBAC: Authorizing `identity:get_group(group_id=59e391a4350244c9888947a0896ee4e2)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:50.930414 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1c09b-6fde-4d39-b4c7-cdfba3588f41 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:50.934073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 217/425] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 7 13:08:50 2022] GET /identity/v3/groups/59e391a4350244c9888947a0896ee4e2 => generated 226 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:50.955641 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1c09b-6fde-4d39-b4c7-cdfba3588f41 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:51.003230 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc0cbc-2771-4077-8fb3-7809e46813f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:51.004235 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc0cbc-2771-4077-8fb3-7809e46813f5 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:51.004697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc0cbc-2771-4077-8fb3-7809e46813f5 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:51.005157 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc0cbc-2771-4077-8fb3-7809e46813f5 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:51.006438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc0cbc-2771-4077-8fb3-7809e46813f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:51.006953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc0cbc-2771-4077-8fb3-7809e46813f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:51.011935 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc0cbc-2771-4077-8fb3-7809e46813f5 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:51.033873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 209/426] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 480 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:51.042484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc0cbc-2771-4077-8fb3-7809e46813f5 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:51.120755 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:51.271484 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1c09b-6fde-4d39-b4c7-cdfba3588f41 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:51.334906 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc0cbc-2771-4077-8fb3-7809e46813f5 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:51.404926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cf270-54ab-4b66-8cfa-6520d1a171f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:51.406609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cf270-54ab-4b66-8cfa-6520d1a171f6 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:51.407476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cf270-54ab-4b66-8cfa-6520d1a171f6 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:51.408675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cf270-54ab-4b66-8cfa-6520d1a171f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:51.410471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041cf270-54ab-4b66-8cfa-6520d1a171f6 None admin] RBAC: Adding query filter params (group.id=59e391a4350244c9888947a0896ee4e2, role.id=fc0a7877dd0f4903a2a6fcc01a829c6a, scope.project.id=1fcdd13492724455ae65d6e80d2f41bc) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:51.411376 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041cf270-54ab-4b66-8cfa-6520d1a171f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:51.414511 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041cf270-54ab-4b66-8cfa-6520d1a171f6 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:51.421557 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4b1ca-c10c-4cde-8acf-74f396ecc4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:51.422559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4b1ca-c10c-4cde-8acf-74f396ecc4a9 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:51.423053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4b1ca-c10c-4cde-8acf-74f396ecc4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:51.423555 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4b1ca-c10c-4cde-8acf-74f396ecc4a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:51.429299 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 218/427] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 7 13:08:50 2022] GET /identity/v3/role_assignments?group.id=59e391a4350244c9888947a0896ee4e2&scope.project.id=1fcdd13492724455ae65d6e80d2f41bc&role.id=fc0a7877dd0f4903a2a6fcc01a829c6a => generated 611 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:08:51.608476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4b1ca-c10c-4cde-8acf-74f396ecc4a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:51.611634 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4b1ca-c10c-4cde-8acf-74f396ecc4a9 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:51.650872 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:51.758262 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780e7ab-ba71-4e4e-bdd9-6a1837c30695 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:51.758665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780e7ab-ba71-4e4e-bdd9-6a1837c30695 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:51.759190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780e7ab-ba71-4e4e-bdd9-6a1837c30695 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:51.762225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 219/428] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:51.771982 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6ea75-88d5-4e9f-9122-c199dc1ee780 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:51.772490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6ea75-88d5-4e9f-9122-c199dc1ee780 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:51.772999 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6ea75-88d5-4e9f-9122-c199dc1ee780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:51.837129 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-13a6ea75-88d5-4e9f-9122-c199dc1ee780 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:51.839281 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-13a6ea75-88d5-4e9f-9122-c199dc1ee780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:51.844211 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 210/429] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:08:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 803 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 13:08:51.854514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4b1ca-c10c-4cde-8acf-74f396ecc4a9 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:51.907430 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:52.002487 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 220/430] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:52.112252 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4b1ca-c10c-4cde-8acf-74f396ecc4a9 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:52.148207 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:52.193012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce58eb-47a9-48a9-b856-5c6a69e6b738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:52.194054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce58eb-47a9-48a9-b856-5c6a69e6b738 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:52.194375 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce58eb-47a9-48a9-b856-5c6a69e6b738 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:52.194650 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce58eb-47a9-48a9-b856-5c6a69e6b738 None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:52.196371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce58eb-47a9-48a9-b856-5c6a69e6b738 None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:52.198637 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce58eb-47a9-48a9-b856-5c6a69e6b738 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:52.203364 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 211/431] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:51 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:08:52.210305 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6ea75-88d5-4e9f-9122-c199dc1ee780 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:52.427946 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6ea75-88d5-4e9f-9122-c199dc1ee780 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:52.506137 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a7f36-b744-4a7d-b489-39d19d8d7a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:52.507674 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a7f36-b744-4a7d-b489-39d19d8d7a21 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:52.508400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a7f36-b744-4a7d-b489-39d19d8d7a21 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:52.509054 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a7f36-b744-4a7d-b489-39d19d8d7a21 None admin] PATH_INFO: `/v3/groups/59e391a4350244c9888947a0896ee4e2` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:52.511544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a7f36-b744-4a7d-b489-39d19d8d7a21 None admin] RBAC: Authorizing `identity:get_group(group_id=59e391a4350244c9888947a0896ee4e2)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:52.514634 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a7f36-b744-4a7d-b489-39d19d8d7a21 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:52.519444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 221/432] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 7 13:08:52 2022] GET /identity/v3/groups/59e391a4350244c9888947a0896ee4e2 => generated 226 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:52.530787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a7f36-b744-4a7d-b489-39d19d8d7a21 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:52.534750 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d6273-458f-4726-b5dc-09be88ab5bd7 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:52.535517 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d6273-458f-4726-b5dc-09be88ab5bd7 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:52.535804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d6273-458f-4726-b5dc-09be88ab5bd7 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:52.538883 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 212/433] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:52.547735 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a6907-ef76-4e3f-8688-8d28e895939b None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:52.549106 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a6907-ef76-4e3f-8688-8d28e895939b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:52.549757 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a6907-ef76-4e3f-8688-8d28e895939b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:52.585561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-499a6907-ef76-4e3f-8688-8d28e895939b None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:52.587536 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-499a6907-ef76-4e3f-8688-8d28e895939b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:52.598503 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:52.696048 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 213/434] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:52.697286 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e1e2b-3aed-49f8-b8e3-b2e698971de6 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:52.697570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e1e2b-3aed-49f8-b8e3-b2e698971de6 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:52.697846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e1e2b-3aed-49f8-b8e3-b2e698971de6 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:52.706032 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 214/435] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:08:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:08:52.706032 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e1e2b-3aed-49f8-b8e3-b2e698971de6 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:52.706344 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a7f36-b744-4a7d-b489-39d19d8d7a21 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:52.748470 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c074dae-2563-49f4-935d-6f70c59617b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:52.749098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c074dae-2563-49f4-935d-6f70c59617b7 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:52.749374 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c074dae-2563-49f4-935d-6f70c59617b7 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:52.750377 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c074dae-2563-49f4-935d-6f70c59617b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:52.750583 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c074dae-2563-49f4-935d-6f70c59617b7 None admin] RBAC: Adding query filter params (group.id=59e391a4350244c9888947a0896ee4e2, role.id=bcc52f73cf03442e922a4757908334de, scope.project.id=920dfa8ad35b4a429aea367ae3067b8e) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:52.750905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c074dae-2563-49f4-935d-6f70c59617b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:52.753097 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c074dae-2563-49f4-935d-6f70c59617b7 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:52.767546 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 222/436] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 7 13:08:52 2022] GET /identity/v3/role_assignments?group.id=59e391a4350244c9888947a0896ee4e2&scope.project.id=920dfa8ad35b4a429aea367ae3067b8e&role.id=bcc52f73cf03442e922a4757908334de => generated 611 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:52.767546 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c4fe4-7e1b-4f73-95df-6d8753d20efb None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:52.767546 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c4fe4-7e1b-4f73-95df-6d8753d20efb None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:52.767546 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c4fe4-7e1b-4f73-95df-6d8753d20efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:52.798420 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-446c4fe4-7e1b-4f73-95df-6d8753d20efb None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:08:52.800081 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-446c4fe4-7e1b-4f73-95df-6d8753d20efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:08:52.934074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 223/437] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:08:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:08:52.961386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e1e2b-3aed-49f8-b8e3-b2e698971de6 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:52.988845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:53.026866 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c9f0a-fd52-4a29-8564-4fee04614882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:53.028035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c9f0a-fd52-4a29-8564-4fee04614882 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:53.028533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c9f0a-fd52-4a29-8564-4fee04614882 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:53.029045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c9f0a-fd52-4a29-8564-4fee04614882 None admin] PATH_INFO: `/v3/roles/member` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:53.050701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c9f0a-fd52-4a29-8564-4fee04614882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:53.059455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-be5c9f0a-fd52-4a29-8564-4fee04614882 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:53.059455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:53.059455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:53.059455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:53.059455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:53.059455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:53.059455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:53.059455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:53.059455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:53.059455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:53.059455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:53.059455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:53.059455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:53.059455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:53.059455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:53.059455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:53.060761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:53.060761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:53.060761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:53.060761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:53.060761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:53.060761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:53.060761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:53.060761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:53.060761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:53.060761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:53.060761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:53.060761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:53.060761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:53.060761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:53.060761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:53.060761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:53.061920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:53.061920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:53.061920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:53.061920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:53.061920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:53.061920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:53.061920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:53.061920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:53.061920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:53.061920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:53.061920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 13:08:53.061920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:53.061920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 215/438] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:53.074700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c4fe4-7e1b-4f73-95df-6d8753d20efb None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:53.124792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:53.219980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c9f0a-fd52-4a29-8564-4fee04614882 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:53.246785 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c4fe4-7e1b-4f73-95df-6d8753d20efb None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:53.308873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ae107-c0f2-41d3-bb95-3e878a6a6c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:53.309961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ae107-c0f2-41d3-bb95-3e878a6a6c22 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:53.310444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ae107-c0f2-41d3-bb95-3e878a6a6c22 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:53.310927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ae107-c0f2-41d3-bb95-3e878a6a6c22 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:53.312244 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ae107-c0f2-41d3-bb95-3e878a6a6c22 None admin] RBAC: Adding query filter params (name=member) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:53.312788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ae107-c0f2-41d3-bb95-3e878a6a6c22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:53.316030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ae107-c0f2-41d3-bb95-3e878a6a6c22 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:53.333351 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 224/439] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:08:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:53.340196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ae107-c0f2-41d3-bb95-3e878a6a6c22 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:53.408861 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c9f0a-fd52-4a29-8564-4fee04614882 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:53.467793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd1066-0d4b-4bbf-8cdd-7d914eb2b4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:53.468456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd1066-0d4b-4bbf-8cdd-7d914eb2b4ca None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:53.468735 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd1066-0d4b-4bbf-8cdd-7d914eb2b4ca None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:53.469006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd1066-0d4b-4bbf-8cdd-7d914eb2b4ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:53.487132 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd1066-0d4b-4bbf-8cdd-7d914eb2b4ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:53.489209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:53.493339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-b4dd1066-0d4b-4bbf-8cdd-7d914eb2b4ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:08:53.493339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:53.493339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:53.493339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:53.493339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:53.493339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:53.493339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:53.493339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:53.493339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:53.493339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:53.493339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:53.493339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:53.493339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:53.493339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:53.493339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:53.493339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:53.495009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:53.495009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:53.495009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:53.495009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:53.495009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:53.495009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:53.495009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:53.495009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:53.495009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:53.495009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:53.495009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:53.495009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:53.495009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:53.495009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:53.495009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:53.495009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:53.496138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:53.496138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:53.496138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:53.496138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:53.496138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:53.496138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:53.496138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:53.496138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:53.496138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:53.496138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:53.496138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:08:53.496138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:53.496138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 216/440] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:08:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:53.501010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd1066-0d4b-4bbf-8cdd-7d914eb2b4ca None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:53.510300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ae107-c0f2-41d3-bb95-3e878a6a6c22 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:53.557718 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11bc0e-86e0-4660-b177-7fac5f300632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:53.558506 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11bc0e-86e0-4660-b177-7fac5f300632 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:53.558795 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11bc0e-86e0-4660-b177-7fac5f300632 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:53.559108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11bc0e-86e0-4660-b177-7fac5f300632 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:53.559764 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:53.658151 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11bc0e-86e0-4660-b177-7fac5f300632 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:53.659897 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11bc0e-86e0-4660-b177-7fac5f300632 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:53.662580 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd1066-0d4b-4bbf-8cdd-7d914eb2b4ca None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:53.719695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a940ff-3f9e-43e1-bc48-aa89fda71b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:53.722388 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a940ff-3f9e-43e1-bc48-aa89fda71b4b None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:53.722692 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a940ff-3f9e-43e1-bc48-aa89fda71b4b None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:53.722968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a940ff-3f9e-43e1-bc48-aa89fda71b4b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:53.735252 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a940ff-3f9e-43e1-bc48-aa89fda71b4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:08:53.742762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-c9a940ff-3f9e-43e1-bc48-aa89fda71b4b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:08:53.742762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:08:53.742762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:08:53.742762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:08:53.742762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:08:53.742762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:08:53.742762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:08:53.742762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:08:53.742762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:08:53.742762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:08:53.742762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:08:53.742762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:08:53.742762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:08:53.742762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:08:53.742762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:08:53.742762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:08:53.744272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:08:53.744272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:08:53.744272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:08:53.744272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:08:53.744272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:08:53.744272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:08:53.744272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:08:53.744272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:08:53.744272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:08:53.744272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:08:53.744272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:08:53.744272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:08:53.744272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:08:53.744272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:08:53.744272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:08:53.744272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:08:53.746599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:08:53.746599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:08:53.746599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:08:53.746599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:08:53.746599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:08:53.746599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:08:53.746599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:08:53.746599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:08:53.746599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:08:53.746599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:08:53.746599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:08:53.746599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:08:53.746599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 217/441] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:08:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:08:53.748338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a940ff-3f9e-43e1-bc48-aa89fda71b4b None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:53.804966 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:53.818378 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 225/442] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:08:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 13:08:53.820600 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11bc0e-86e0-4660-b177-7fac5f300632 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:53.874926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:53.882126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a940ff-3f9e-43e1-bc48-aa89fda71b4b None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:53.940599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c8fd8-77a5-4abf-ab93-6c8bf3bd06d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:53.941766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c8fd8-77a5-4abf-ab93-6c8bf3bd06d1 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:53.942084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c8fd8-77a5-4abf-ab93-6c8bf3bd06d1 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:53.943245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c8fd8-77a5-4abf-ab93-6c8bf3bd06d1 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:53.943761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c8fd8-77a5-4abf-ab93-6c8bf3bd06d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:53.944252 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c8fd8-77a5-4abf-ab93-6c8bf3bd06d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:53.946638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c8fd8-77a5-4abf-ab93-6c8bf3bd06d1 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:53.961762 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11bc0e-86e0-4660-b177-7fac5f300632 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:53.962096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 218/443] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:08:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:53.963708 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c8fd8-77a5-4abf-ab93-6c8bf3bd06d1 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:54.019945 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b7fbe-bf1a-41e8-8ebe-b5a5ad460a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:54.020556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b7fbe-bf1a-41e8-8ebe-b5a5ad460a38 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:54.020904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b7fbe-bf1a-41e8-8ebe-b5a5ad460a38 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:54.021147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b7fbe-bf1a-41e8-8ebe-b5a5ad460a38 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:54.021898 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b7fbe-bf1a-41e8-8ebe-b5a5ad460a38 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:54.022213 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b7fbe-bf1a-41e8-8ebe-b5a5ad460a38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:54.024001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b7fbe-bf1a-41e8-8ebe-b5a5ad460a38 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:54.034476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 226/444] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:08:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:54.035489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b7fbe-bf1a-41e8-8ebe-b5a5ad460a38 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:54.114577 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:54.132220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c8fd8-77a5-4abf-ab93-6c8bf3bd06d1 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:54.182335 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4433774-2fec-4933-9e9e-6c5ae1500987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:54.183029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4433774-2fec-4933-9e9e-6c5ae1500987 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:54.183308 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4433774-2fec-4933-9e9e-6c5ae1500987 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:54.183587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4433774-2fec-4933-9e9e-6c5ae1500987 None admin] PATH_INFO: `/v3/projects/1fcdd13492724455ae65d6e80d2f41bc` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:54.184892 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4433774-2fec-4933-9e9e-6c5ae1500987 None admin] RBAC: Authorizing `identity:get_project(project_id=1fcdd13492724455ae65d6e80d2f41bc)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:54.186365 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4433774-2fec-4933-9e9e-6c5ae1500987 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:54.188257 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 219/445] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:53 2022] GET /identity/v3/projects/1fcdd13492724455ae65d6e80d2f41bc => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:54.189564 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4433774-2fec-4933-9e9e-6c5ae1500987 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:54.217382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b7fbe-bf1a-41e8-8ebe-b5a5ad460a38 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:54.237896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:54.279115 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92eb46-73ef-47a2-b46f-dd9a98b73acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:54.280153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92eb46-73ef-47a2-b46f-dd9a98b73acb None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:54.280689 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92eb46-73ef-47a2-b46f-dd9a98b73acb None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:54.281255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92eb46-73ef-47a2-b46f-dd9a98b73acb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:54.362571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4433774-2fec-4933-9e9e-6c5ae1500987 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:54.408333 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff115620-1d55-4bc8-aaa9-79d4f8777c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:54.409380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff115620-1d55-4bc8-aaa9-79d4f8777c80 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:54.409825 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff115620-1d55-4bc8-aaa9-79d4f8777c80 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:54.410412 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff115620-1d55-4bc8-aaa9-79d4f8777c80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:54.417133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92eb46-73ef-47a2-b46f-dd9a98b73acb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:54.418959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92eb46-73ef-47a2-b46f-dd9a98b73acb None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:54.505447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:54.526290 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff115620-1d55-4bc8-aaa9-79d4f8777c80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:54.529375 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff115620-1d55-4bc8-aaa9-79d4f8777c80 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:54.581617 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:54.586399 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 227/446] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:08:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:08:54.587469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92eb46-73ef-47a2-b46f-dd9a98b73acb None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:54.725827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 220/447] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:08:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:08:54.726800 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff115620-1d55-4bc8-aaa9-79d4f8777c80 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:54.750289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92eb46-73ef-47a2-b46f-dd9a98b73acb None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:54.808602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb58627-101c-4fd6-b097-5b7823654ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:54.809222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb58627-101c-4fd6-b097-5b7823654ff1 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:54.809536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb58627-101c-4fd6-b097-5b7823654ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:54.809824 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb58627-101c-4fd6-b097-5b7823654ff1 None admin] PATH_INFO: `/v3/groups/59e391a4350244c9888947a0896ee4e2` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:54.811261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb58627-101c-4fd6-b097-5b7823654ff1 None admin] RBAC: Authorizing `identity:get_group(group_id=59e391a4350244c9888947a0896ee4e2)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:54.812929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb58627-101c-4fd6-b097-5b7823654ff1 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:54.816117 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 228/448] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 7 13:08:54 2022] GET /identity/v3/groups/59e391a4350244c9888947a0896ee4e2 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:54.817700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb58627-101c-4fd6-b097-5b7823654ff1 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:54.861300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:54.900356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff115620-1d55-4bc8-aaa9-79d4f8777c80 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:54.950530 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:54.967913 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d30d8-dc05-4b74-a379-be3795e0fef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:54.968878 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d30d8-dc05-4b74-a379-be3795e0fef5 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:54.969364 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d30d8-dc05-4b74-a379-be3795e0fef5 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:54.969852 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d30d8-dc05-4b74-a379-be3795e0fef5 None admin] PATH_INFO: `/v3/projects/0ee134892b8643809999afc672e86085` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:54.972178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d30d8-dc05-4b74-a379-be3795e0fef5 None admin] RBAC: Authorizing `identity:get_project(project_id=0ee134892b8643809999afc672e86085)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:54.974882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d30d8-dc05-4b74-a379-be3795e0fef5 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:54.985204 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 221/449] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:54 2022] GET /identity/v3/projects/0ee134892b8643809999afc672e86085 => generated 344 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:54.985204 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d30d8-dc05-4b74-a379-be3795e0fef5 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:54.985635 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb58627-101c-4fd6-b097-5b7823654ff1 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:55.046459 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226bb0df-f200-451f-8175-e41a516b8c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:55.047158 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bb0df-f200-451f-8175-e41a516b8c9c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:55.047488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bb0df-f200-451f-8175-e41a516b8c9c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:55.047765 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bb0df-f200-451f-8175-e41a516b8c9c None admin] PATH_INFO: `/v3/projects/920dfa8ad35b4a429aea367ae3067b8e` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:55.049831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bb0df-f200-451f-8175-e41a516b8c9c None admin] RBAC: Authorizing `identity:get_project(project_id=920dfa8ad35b4a429aea367ae3067b8e)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:55.051249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bb0df-f200-451f-8175-e41a516b8c9c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:55.053202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 229/450] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:08:54 2022] GET /identity/v3/projects/920dfa8ad35b4a429aea367ae3067b8e => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:55.054634 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226bb0df-f200-451f-8175-e41a516b8c9c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:55.158631 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d30d8-dc05-4b74-a379-be3795e0fef5 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:55.188185 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:55.205310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226bb0df-f200-451f-8175-e41a516b8c9c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:55.207738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db0f77-bf23-4b04-b323-e3d48698b953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:55.208413 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db0f77-bf23-4b04-b323-e3d48698b953 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:55.208703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db0f77-bf23-4b04-b323-e3d48698b953 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:55.209108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db0f77-bf23-4b04-b323-e3d48698b953 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:55.209861 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db0f77-bf23-4b04-b323-e3d48698b953 None admin] RBAC: Adding query filter params (group.id=59e391a4350244c9888947a0896ee4e2, role.id=bcc52f73cf03442e922a4757908334de, scope.project.id=1fcdd13492724455ae65d6e80d2f41bc) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:55.210038 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db0f77-bf23-4b04-b323-e3d48698b953 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:55.211545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db0f77-bf23-4b04-b323-e3d48698b953 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:55.218301 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 222/451] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 7 13:08:54 2022] GET /identity/v3/role_assignments?group.id=59e391a4350244c9888947a0896ee4e2&scope.project.id=1fcdd13492724455ae65d6e80d2f41bc&role.id=bcc52f73cf03442e922a4757908334de => generated 611 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:55.219540 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db0f77-bf23-4b04-b323-e3d48698b953 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:55.263860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9178fbf-7eb1-48d8-ab28-b87fc213da40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:55.264686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9178fbf-7eb1-48d8-ab28-b87fc213da40 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:55.265088 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9178fbf-7eb1-48d8-ab28-b87fc213da40 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:55.265581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9178fbf-7eb1-48d8-ab28-b87fc213da40 None admin] PATH_INFO: `/v3/groups/ef70b838a8bf4e6e871179b8b339e868` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:55.268289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9178fbf-7eb1-48d8-ab28-b87fc213da40 None admin] RBAC: Authorizing `identity:get_group(group_id=ef70b838a8bf4e6e871179b8b339e868)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:55.270379 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9178fbf-7eb1-48d8-ab28-b87fc213da40 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:55.279404 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 230/452] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 7 13:08:55 2022] GET /identity/v3/groups/ef70b838a8bf4e6e871179b8b339e868 => generated 229 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:55.281435 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9178fbf-7eb1-48d8-ab28-b87fc213da40 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:55.351281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:55.439617 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db0f77-bf23-4b04-b323-e3d48698b953 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:55.495561 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9178fbf-7eb1-48d8-ab28-b87fc213da40 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:55.512877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a0559-1393-455c-9515-6a062c83e69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:55.514037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a0559-1393-455c-9515-6a062c83e69d None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:55.514498 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a0559-1393-455c-9515-6a062c83e69d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:55.515031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a0559-1393-455c-9515-6a062c83e69d None admin] PATH_INFO: `/v3/groups/59e391a4350244c9888947a0896ee4e2` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:55.518625 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a0559-1393-455c-9515-6a062c83e69d None admin] RBAC: Authorizing `identity:get_group(group_id=59e391a4350244c9888947a0896ee4e2)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:55.521879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a0559-1393-455c-9515-6a062c83e69d None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:55.525377 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 223/453] 10.222.0.62 () {56 vars in 1180 bytes} [Wed Sep 7 13:08:55 2022] GET /identity/v3/groups/59e391a4350244c9888947a0896ee4e2 => generated 226 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:08:55.537673 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a0559-1393-455c-9515-6a062c83e69d None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:08:55.574142 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bd5c7-cc9e-47f7-b346-71a7f502defa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:55.575020 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bd5c7-cc9e-47f7-b346-71a7f502defa None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:55.575020 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bd5c7-cc9e-47f7-b346-71a7f502defa None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:55.575428 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bd5c7-cc9e-47f7-b346-71a7f502defa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:55.576429 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6bd5c7-cc9e-47f7-b346-71a7f502defa None admin] RBAC: Adding query filter params (group.id=ef70b838a8bf4e6e871179b8b339e868, role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=0ee134892b8643809999afc672e86085) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:55.576881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6bd5c7-cc9e-47f7-b346-71a7f502defa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:55.580366 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6bd5c7-cc9e-47f7-b346-71a7f502defa None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:55.589741 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 231/454] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 7 13:08:55 2022] GET /identity/v3/role_assignments?group.id=ef70b838a8bf4e6e871179b8b339e868&scope.project.id=0ee134892b8643809999afc672e86085&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 611 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:08:55.751695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a0559-1393-455c-9515-6a062c83e69d None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:08:55.806732 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:08:55.824871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e66be2-ec67-4c58-aa07-a357bb06168d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:08:55.825937 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e66be2-ec67-4c58-aa07-a357bb06168d None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:08:55.826379 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e66be2-ec67-4c58-aa07-a357bb06168d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:08:55.826809 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e66be2-ec67-4c58-aa07-a357bb06168d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:08:55.828383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e66be2-ec67-4c58-aa07-a357bb06168d None admin] RBAC: Adding query filter params (group.id=59e391a4350244c9888947a0896ee4e2, role.id=fc0a7877dd0f4903a2a6fcc01a829c6a, scope.project.id=920dfa8ad35b4a429aea367ae3067b8e) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:08:55.828950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e66be2-ec67-4c58-aa07-a357bb06168d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:08:55.833125 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e66be2-ec67-4c58-aa07-a357bb06168d None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:08:55.845813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 224/455] 10.222.0.62 () {56 vars in 1376 bytes} [Wed Sep 7 13:08:55 2022] GET /identity/v3/role_assignments?group.id=59e391a4350244c9888947a0896ee4e2&scope.project.id=920dfa8ad35b4a429aea367ae3067b8e&role.id=fc0a7877dd0f4903a2a6fcc01a829c6a => generated 611 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:09:00.523455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15872cd-8dc4-4461-99e8-b42f6cee5df8 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:00.524225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15872cd-8dc4-4461-99e8-b42f6cee5df8 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:00.524395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15872cd-8dc4-4461-99e8-b42f6cee5df8 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:00.527613 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 232/456] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:00.536750 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c1be3-44c2-43a3-b14b-eb9d5785f4ae None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:00.537559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c1be3-44c2-43a3-b14b-eb9d5785f4ae None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:00.537692 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c1be3-44c2-43a3-b14b-eb9d5785f4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:00.573007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-2f2c1be3-44c2-43a3-b14b-eb9d5785f4ae None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:00.575235 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2f2c1be3-44c2-43a3-b14b-eb9d5785f4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:00.748693 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 225/457] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:00.819689 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4900abe-396f-4a1c-b0a2-eb3add036331 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:00.820177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4900abe-396f-4a1c-b0a2-eb3add036331 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:00.820662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4900abe-396f-4a1c-b0a2-eb3add036331 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:00.823883 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 233/458] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:00.832003 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cf392-922b-4a07-80a0-e8d805f033af None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:00.832565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cf392-922b-4a07-80a0-e8d805f033af None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:00.832887 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cf392-922b-4a07-80a0-e8d805f033af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:00.875070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-970cf392-922b-4a07-80a0-e8d805f033af None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:00.878026 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-970cf392-922b-4a07-80a0-e8d805f033af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:00.922660 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b2050-6006-4017-9362-3911229d790b None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:00.923745 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b2050-6006-4017-9362-3911229d790b None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:00.924226 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b2050-6006-4017-9362-3911229d790b None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:00.931198 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 234/459] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:09:00.937282 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99890b-4872-4abd-b0ac-c8bce5f65d24 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:00.937753 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99890b-4872-4abd-b0ac-c8bce5f65d24 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:00.938190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99890b-4872-4abd-b0ac-c8bce5f65d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:01.010579 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-ea99890b-4872-4abd-b0ac-c8bce5f65d24 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:01.011929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ea99890b-4872-4abd-b0ac-c8bce5f65d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:01.061483 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 226/460] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:01.079008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:01.243148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 235/461] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:01.484100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970cf392-922b-4a07-80a0-e8d805f033af None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:01.507021 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6bc3d-0eb5-4ef1-b467-0424491c22a2 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:01.507521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6bc3d-0eb5-4ef1-b467-0424491c22a2 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:01.507999 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b6bc3d-0eb5-4ef1-b467-0424491c22a2 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:01.515382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 236/462] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:09:01.518214 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8037a62-39b0-46bd-9de4-8b0b9589d9e3 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:01.518699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8037a62-39b0-46bd-9de4-8b0b9589d9e3 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:01.519206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8037a62-39b0-46bd-9de4-8b0b9589d9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:01.584248 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b8037a62-39b0-46bd-9de4-8b0b9589d9e3 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:01.587710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b8037a62-39b0-46bd-9de4-8b0b9589d9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:01.599018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:01.710848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970cf392-922b-4a07-80a0-e8d805f033af None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:01.735361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 237/463] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:09:01.737410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8037a62-39b0-46bd-9de4-8b0b9589d9e3 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:01.854854 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86d7e3-d05a-42f0-b41f-123aa92049fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:01.856367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86d7e3-d05a-42f0-b41f-123aa92049fe None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:01.857076 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86d7e3-d05a-42f0-b41f-123aa92049fe None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:01.857756 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86d7e3-d05a-42f0-b41f-123aa92049fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:01.885484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86d7e3-d05a-42f0-b41f-123aa92049fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:01.888387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86d7e3-d05a-42f0-b41f-123aa92049fe None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:01.922178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-bd86d7e3-d05a-42f0-b41f-123aa92049fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:01.922178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:01.922178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:01.922178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:01.922178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:01.922178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:01.922178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:01.922178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:01.922178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:01.922178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:01.922178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:01.922178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:01.922178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:01.922178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:01.922178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:01.922178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:01.924869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:01.924869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:01.924869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:01.924869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:01.924869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:01.924869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:01.924869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:01.924869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:01.924869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:01.924869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:01.924869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:01.924869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:01.924869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:01.924869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:01.924869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:01.924869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:01.924869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:01.926839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:01.926839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:01.926839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:01.927345 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:01.929768 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 227/464] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 446 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:01.934459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86d7e3-d05a-42f0-b41f-123aa92049fe None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:01.957390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8037a62-39b0-46bd-9de4-8b0b9589d9e3 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:02.011601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f0ab8-f41d-45ec-8734-05fb4d700cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:02.012556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f0ab8-f41d-45ec-8734-05fb4d700cbf None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:02.013130 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f0ab8-f41d-45ec-8734-05fb4d700cbf None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:02.013699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f0ab8-f41d-45ec-8734-05fb4d700cbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:02.035709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f0ab8-f41d-45ec-8734-05fb4d700cbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:02.038738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f0ab8-f41d-45ec-8734-05fb4d700cbf None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:02.072215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-962f0ab8-f41d-45ec-8734-05fb4d700cbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:02.072215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:02.072215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:02.072215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:02.072215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:02.072215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:02.072215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:02.072215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:02.072215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:02.072215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:02.072215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:02.072215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:02.072215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:02.072215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:02.072215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:02.072215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:02.074005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:02.074005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:02.074005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:02.074005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:02.074005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:02.074005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:02.074005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:02.074005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:02.074005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:02.074005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:02.074005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:02.074005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:02.074005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:02.074005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:02.074005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:02.074005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:02.074005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:02.075672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:02.075672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:02.075672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:02.083016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 238/465] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:02.083016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f0ab8-f41d-45ec-8734-05fb4d700cbf None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:02.147503 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86d7e3-d05a-42f0-b41f-123aa92049fe None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:02.150940 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:02.219986 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd801f74-e517-4c03-9229-2183067b93cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:02.221608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd801f74-e517-4c03-9229-2183067b93cc None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:02.222125 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd801f74-e517-4c03-9229-2183067b93cc None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:02.222581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd801f74-e517-4c03-9229-2183067b93cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:02.245392 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd801f74-e517-4c03-9229-2183067b93cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:02.250165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd801f74-e517-4c03-9229-2183067b93cc None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:02.281649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-bd801f74-e517-4c03-9229-2183067b93cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:02.281649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:02.281649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:02.281649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:02.281649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:02.281649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:02.281649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:02.281649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:02.281649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:02.281649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:02.281649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:02.281649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:02.281649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:02.281649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:02.281649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:02.281649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:02.284018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:02.284018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:02.284018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:02.284018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:02.284018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:02.284018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:02.284018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:02.284018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:02.284018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:02.284018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:02.284018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:02.284018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:02.284018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:02.284018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:02.284018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:02.284018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:02.284018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:02.287374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:02.287374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:02.287374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:02.287374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 228/466] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:02.287374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd801f74-e517-4c03-9229-2183067b93cc None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:02.303123 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f0ab8-f41d-45ec-8734-05fb4d700cbf None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:02.366165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:02.377011 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587f33c-d3c1-4f10-a124-cb61a1f77127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:02.378171 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587f33c-d3c1-4f10-a124-cb61a1f77127 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:02.378663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587f33c-d3c1-4f10-a124-cb61a1f77127 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:02.379269 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587f33c-d3c1-4f10-a124-cb61a1f77127 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:02.380545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587f33c-d3c1-4f10-a124-cb61a1f77127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:02.381114 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587f33c-d3c1-4f10-a124-cb61a1f77127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:02.383848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587f33c-d3c1-4f10-a124-cb61a1f77127 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:02.406920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 239/467] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:02.408596 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587f33c-d3c1-4f10-a124-cb61a1f77127 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:02.503416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd801f74-e517-4c03-9229-2183067b93cc None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:02.572100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1824b5-5676-48f1-8052-ef63be7b5139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:02.573023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1824b5-5676-48f1-8052-ef63be7b5139 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:02.573518 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1824b5-5676-48f1-8052-ef63be7b5139 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:02.574005 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1824b5-5676-48f1-8052-ef63be7b5139 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:02.575234 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1824b5-5676-48f1-8052-ef63be7b5139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:02.575780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1824b5-5676-48f1-8052-ef63be7b5139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:02.576735 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587f33c-d3c1-4f10-a124-cb61a1f77127 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:02.578387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1824b5-5676-48f1-8052-ef63be7b5139 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:02.603652 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 229/468] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:02.613301 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1824b5-5676-48f1-8052-ef63be7b5139 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:02.637434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:02.652778 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed780997-a910-400c-b20e-50c97d6387b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:02.653839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed780997-a910-400c-b20e-50c97d6387b3 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:02.654325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed780997-a910-400c-b20e-50c97d6387b3 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:02.654813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed780997-a910-400c-b20e-50c97d6387b3 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:02.656075 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed780997-a910-400c-b20e-50c97d6387b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:02.656622 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed780997-a910-400c-b20e-50c97d6387b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:02.660000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed780997-a910-400c-b20e-50c97d6387b3 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:02.683612 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 240/469] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:02.685726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e826a5-8201-46ed-9d94-70a5b49f1c6e None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:02.686172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e826a5-8201-46ed-9d94-70a5b49f1c6e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:02.686605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e826a5-8201-46ed-9d94-70a5b49f1c6e None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:02.689452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 241/470] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:02.690766 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e826a5-8201-46ed-9d94-70a5b49f1c6e None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:02.803869 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:02.808702 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1824b5-5676-48f1-8052-ef63be7b5139 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:02.877061 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35edfa-bb51-4d95-a7b2-96b4d64e73d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:02.878002 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35edfa-bb51-4d95-a7b2-96b4d64e73d1 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:02.878478 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35edfa-bb51-4d95-a7b2-96b4d64e73d1 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:02.878943 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35edfa-bb51-4d95-a7b2-96b4d64e73d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:02.898896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee35edfa-bb51-4d95-a7b2-96b4d64e73d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:02.901761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee35edfa-bb51-4d95-a7b2-96b4d64e73d1 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:02.904561 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e826a5-8201-46ed-9d94-70a5b49f1c6e None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:02.931260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-ee35edfa-bb51-4d95-a7b2-96b4d64e73d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:02.931260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:02.931260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:02.931260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:02.931260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:02.931260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:02.931260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:02.931260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:02.931260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:02.931260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:02.931260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:02.931260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:02.931260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:02.931260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:02.931260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:02.931260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:02.932802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:02.932802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:02.932802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:02.932802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:02.932802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:02.932802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:02.932802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:02.932802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:02.932802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:02.932802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:02.932802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:02.932802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:02.932802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:02.932802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:02.932802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:02.932802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:02.932802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:02.934661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:02.934661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:02.934661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:02.934661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 230/471] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:02.935136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35edfa-bb51-4d95-a7b2-96b4d64e73d1 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:02.963481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c30f73-b665-4eb3-9f57-c0cdb0d7308f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:02.964541 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c30f73-b665-4eb3-9f57-c0cdb0d7308f None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:02.965188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c30f73-b665-4eb3-9f57-c0cdb0d7308f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:02.965720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c30f73-b665-4eb3-9f57-c0cdb0d7308f None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:02.966771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c30f73-b665-4eb3-9f57-c0cdb0d7308f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:02.969521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c30f73-b665-4eb3-9f57-c0cdb0d7308f None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:03.074818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:03.124081 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 242/472] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:09:02 2022] POST /identity/v3/users => generated 248 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:09:03.124081 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c30f73-b665-4eb3-9f57-c0cdb0d7308f None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:03.155735 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35edfa-bb51-4d95-a7b2-96b4d64e73d1 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:03.238082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec757d-bc80-4cc4-a1c1-41c0ea718970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:03.239473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec757d-bc80-4cc4-a1c1-41c0ea718970 None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:03.239983 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec757d-bc80-4cc4-a1c1-41c0ea718970 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:03.240474 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec757d-bc80-4cc4-a1c1-41c0ea718970 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:03.241765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec757d-bc80-4cc4-a1c1-41c0ea718970 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:03.244780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec757d-bc80-4cc4-a1c1-41c0ea718970 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:03.373783 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c30f73-b665-4eb3-9f57-c0cdb0d7308f None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:03.393822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:03.449537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc29823-9371-42b3-ad3d-54844ec18056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:03.449537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc29823-9371-42b3-ad3d-54844ec18056 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:03.450022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc29823-9371-42b3-ad3d-54844ec18056 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:03.450689 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc29823-9371-42b3-ad3d-54844ec18056 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:03.451933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc29823-9371-42b3-ad3d-54844ec18056 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:03.456230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc29823-9371-42b3-ad3d-54844ec18056 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:03.456520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 231/473] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:09:02 2022] POST /identity/v3/users => generated 248 bytes in 521 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:09:03.459807 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5abfe-9ee6-47c3-aaf2-ff9f8a904671 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:03.460291 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5abfe-9ee6-47c3-aaf2-ff9f8a904671 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:03.460732 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5abfe-9ee6-47c3-aaf2-ff9f8a904671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:03.497503 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-62b5abfe-9ee6-47c3-aaf2-ff9f8a904671 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:03.500471 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-62b5abfe-9ee6-47c3-aaf2-ff9f8a904671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:03.590705 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 243/474] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:09:03 2022] POST /identity/v3/users => generated 246 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:09:03.593129 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc29823-9371-42b3-ad3d-54844ec18056 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:03.663162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 232/475] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:03.844900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc29823-9371-42b3-ad3d-54844ec18056 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:03.927174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed993ec-552e-474c-8452-3b5603909bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:03.928489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed993ec-552e-474c-8452-3b5603909bee None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:03.928996 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed993ec-552e-474c-8452-3b5603909bee None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:03.929494 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed993ec-552e-474c-8452-3b5603909bee None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:03.930811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed993ec-552e-474c-8452-3b5603909bee None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:03.931402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed993ec-552e-474c-8452-3b5603909bee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:03.934159 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed993ec-552e-474c-8452-3b5603909bee None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:03.965504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 244/476] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:03.974609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5abfe-9ee6-47c3-aaf2-ff9f8a904671 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:04.072949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:04.149961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5abfe-9ee6-47c3-aaf2-ff9f8a904671 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:04.206479 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859758bd-af49-437d-b7b2-960bc9646c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:04.207481 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859758bd-af49-437d-b7b2-960bc9646c1f None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:04.207781 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859758bd-af49-437d-b7b2-960bc9646c1f None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:04.208158 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859758bd-af49-437d-b7b2-960bc9646c1f None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:04.209180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859758bd-af49-437d-b7b2-960bc9646c1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:04.210708 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859758bd-af49-437d-b7b2-960bc9646c1f None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:04.297005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed993ec-552e-474c-8452-3b5603909bee None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:04.355931 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:04.429122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 233/477] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:09:03 2022] POST /identity/v3/users => generated 249 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:09:04.551224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed993ec-552e-474c-8452-3b5603909bee None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:04.631322 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47293e14-5f45-4a87-800a-355a97329364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:04.632176 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47293e14-5f45-4a87-800a-355a97329364 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:04.632454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47293e14-5f45-4a87-800a-355a97329364 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:04.632747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47293e14-5f45-4a87-800a-355a97329364 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:04.633454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47293e14-5f45-4a87-800a-355a97329364 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:04.634939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47293e14-5f45-4a87-800a-355a97329364 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:04.722563 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.sql.core [None req-47293e14-5f45-4a87-800a-355a97329364 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 13:09:04.722563 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 07 13:09:04.722563 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [parameters: {'id': '92d0f211c6314ca5ad14b4807c3a4bec', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 07 13:09:04.722563 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=105315) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 13:09:04.746848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-47293e14-5f45-4a87-800a-355a97329364 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 07 13:09:04.746848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:04.746848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 13:09:04.746848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 13:09:04.746848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 13:09:04.746848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 13:09:04.746848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 13:09:04.746848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 13:09:04.746848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 13:09:04.746848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application conn.query(q) Sep 07 13:09:04.746848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 13:09:04.746848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 13:09:04.746848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 13:09:04.746848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application result.read() Sep 07 13:09:04.746848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return ref.to_dict() Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 13:09:04.748639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application next(self.gen) Sep 07 13:09:04.750463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 13:09:04.750463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application yield resource Sep 07 13:09:04.750463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 13:09:04.750463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application next(self.gen) Sep 07 13:09:04.750463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 13:09:04.750463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 13:09:04.750463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 13:09:04.750463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 13:09:04.750463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 13:09:04.750463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise self.value Sep 07 13:09:04.750463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 13:09:04.750463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 13:09:04.750463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 13:09:04.750463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application session.commit() Sep 07 13:09:04.750463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 13:09:04.750463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 13:09:04.750463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 13:09:04.754301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 13:09:04.754301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 13:09:04.754301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self.session.flush() Sep 07 13:09:04.754301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 13:09:04.754301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 13:09:04.754301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 13:09:04.754301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 13:09:04.754301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 13:09:04.754301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application compat.raise_( Sep 07 13:09:04.754301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 13:09:04.754301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception Sep 07 13:09:04.754301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 13:09:04.754301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 13:09:04.754301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 13:09:04.754301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 13:09:04.754301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 13:09:04.754301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 13:09:04.757790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 13:09:04.757790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 13:09:04.757790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 07 13:09:04.757790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 07 13:09:04.757790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 13:09:04.757790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 13:09:04.757790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 13:09:04.757790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 13:09:04.757790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 13:09:04.757790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 13:09:04.757790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 13:09:04.757790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 13:09:04.757790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 13:09:04.757790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 13:09:04.757790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 13:09:04.757790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception Sep 07 13:09:04.765206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 13:09:04.765206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 13:09:04.765206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 13:09:04.765206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 13:09:04.765206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 13:09:04.765206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 13:09:04.765206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 13:09:04.765206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application conn.query(q) Sep 07 13:09:04.765206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 13:09:04.765206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 13:09:04.765206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 13:09:04.765206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application result.read() Sep 07 13:09:04.765206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 13:09:04.765206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 13:09:04.765206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 13:09:04.765206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 13:09:04.769398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 13:09:04.769398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 13:09:04.769398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 13:09:04.769398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 13:09:04.769398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 13:09:04.769398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 07 13:09:04.769398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [parameters: {'id': '92d0f211c6314ca5ad14b4807c3a4bec', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 07 13:09:04.769398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 13:09:04.769398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:04.769398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:09:04.769398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:04.769398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:04.769398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:04.769398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:04.769398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:04.769398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:04.772595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:04.772595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:04.772595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:04.772595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:04.772595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:04.772595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:04.772595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 07 13:09:04.772595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 07 13:09:04.772595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:04.772595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:04.772595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 07 13:09:04.772595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 07 13:09:04.772595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 13:09:04.772595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 13:09:04.772595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 07 13:09:04.772595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:04.777366 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 245/478] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Sep 7 13:09:04 2022] POST /identity/v3/roles => generated 144 bytes in 453 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:04.777604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859758bd-af49-437d-b7b2-960bc9646c1f None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:05.004036 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859758bd-af49-437d-b7b2-960bc9646c1f None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:05.024169 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:05.091971 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06f637-721d-465a-811c-325e542e8bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:05.091971 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06f637-721d-465a-811c-325e542e8bc1 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:05.091971 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06f637-721d-465a-811c-325e542e8bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:05.091971 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06f637-721d-465a-811c-325e542e8bc1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:05.124822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06f637-721d-465a-811c-325e542e8bc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:05.137818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-ed06f637-721d-465a-811c-325e542e8bc1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:09:05.137818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:05.137818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:05.137818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:05.137818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:05.137818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:05.137818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:05.137818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:05.137818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:05.137818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:05.137818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:05.137818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:05.137818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:05.137818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:05.137818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:05.137818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:05.139607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:05.139607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:05.139607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:05.139607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:05.139607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:05.139607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:05.139607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:05.139607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:05.139607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:05.139607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:05.139607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:05.139607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:05.139607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:05.139607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:05.139607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:05.139607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:05.140495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:05.140495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:05.140495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:05.140495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:05.140495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:05.140495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:05.140495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:05.140495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:05.140495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:05.140495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:05.140495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 13:09:05.140495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:05.140495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 234/479] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:09:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:05.150155 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47293e14-5f45-4a87-800a-355a97329364 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:05.517761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:05.543310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47293e14-5f45-4a87-800a-355a97329364 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:05.633421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f8d2f-84e0-4e85-9e5e-967bb0b30587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:05.634527 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f8d2f-84e0-4e85-9e5e-967bb0b30587 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:05.635033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f8d2f-84e0-4e85-9e5e-967bb0b30587 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:05.635516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f8d2f-84e0-4e85-9e5e-967bb0b30587 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:05.637248 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f8d2f-84e0-4e85-9e5e-967bb0b30587 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:05.644211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f8d2f-84e0-4e85-9e5e-967bb0b30587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:05.647023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f8d2f-84e0-4e85-9e5e-967bb0b30587 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:05.669630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 246/480] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 520 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:07.703705 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e399a5-d6d5-499f-9d4d-22a899a21c4c None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:07.704983 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e399a5-d6d5-499f-9d4d-22a899a21c4c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:07.705135 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e399a5-d6d5-499f-9d4d-22a899a21c4c None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:07.707839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 235/481] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:07.721708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929c1f1-7278-46b5-a434-128e678b5439 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:07.722504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929c1f1-7278-46b5-a434-128e678b5439 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:07.722637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929c1f1-7278-46b5-a434-128e678b5439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:07.773002 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-0929c1f1-7278-46b5-a434-128e678b5439 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:07.775639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0929c1f1-7278-46b5-a434-128e678b5439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:07.920759 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504aa30f-b3fe-4f98-bd87-ea61ca9b5878 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:07.921277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504aa30f-b3fe-4f98-bd87-ea61ca9b5878 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:07.921659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504aa30f-b3fe-4f98-bd87-ea61ca9b5878 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:07.936822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 236/482] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:07 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:07.948790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a01e7c-e51b-4b53-bc36-af91e3aeadbd None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:07.949239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a01e7c-e51b-4b53-bc36-af91e3aeadbd None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:07.949692 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a01e7c-e51b-4b53-bc36-af91e3aeadbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:07.982221 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 247/483] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:09:08.033082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-b0a01e7c-e51b-4b53-bc36-af91e3aeadbd None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:08.035377 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b0a01e7c-e51b-4b53-bc36-af91e3aeadbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:08.272412 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 237/484] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:08.636403 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ab69e-df51-4beb-bcd9-5bb20001d389 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:08.636692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ab69e-df51-4beb-bcd9-5bb20001d389 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:08.637074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ab69e-df51-4beb-bcd9-5bb20001d389 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:08.638808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 248/485] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:08.646016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a599a0-b406-4c42-9bf7-b4a4f9a696c9 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:08.646449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a599a0-b406-4c42-9bf7-b4a4f9a696c9 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:08.646918 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a599a0-b406-4c42-9bf7-b4a4f9a696c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:08.690537 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-f2a599a0-b406-4c42-9bf7-b4a4f9a696c9 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:08.693172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f2a599a0-b406-4c42-9bf7-b4a4f9a696c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:08.772962 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ab69e-df51-4beb-bcd9-5bb20001d389 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:08.888592 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 238/486] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:08.982706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a599a0-b406-4c42-9bf7-b4a4f9a696c9 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:08.993763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:09.044330 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ab69e-df51-4beb-bcd9-5bb20001d389 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:09.053057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:09.142807 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c0714-ba85-4bc0-848c-0a1d40b3330a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:09.144516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c0714-ba85-4bc0-848c-0a1d40b3330a None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:09.146155 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c0714-ba85-4bc0-848c-0a1d40b3330a None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:09.147120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c0714-ba85-4bc0-848c-0a1d40b3330a None admin] PATH_INFO: `/v3/roles/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:09.175318 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c0714-ba85-4bc0-848c-0a1d40b3330a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:09.187449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-cd8c0714-ba85-4bc0-848c-0a1d40b3330a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:09.187449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:09.187449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:09.187449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:09.187449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:09.187449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:09.187449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:09.187449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:09.187449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:09.187449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:09.187449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:09.187449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:09.187449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:09.187449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:09.187449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:09.187449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:09.189312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:09.189312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:09.189312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:09.189312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:09.189312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:09.189312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:09.189312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:09.189312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:09.189312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:09.189312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:09.189312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:09.189312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:09.189312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:09.189312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:09.189312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:09.189312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:09.190878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:09.190878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:09.190878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:09.190878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:09.190878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:09.190878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:09.190878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:09.190878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:09.190878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:09.190878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:09.190878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:09.190878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:09.196498 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 249/487] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:09.207496 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c0714-ba85-4bc0-848c-0a1d40b3330a None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:09.282078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a599a0-b406-4c42-9bf7-b4a4f9a696c9 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:09.358864 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c0e36-926f-481c-bad1-3dde61a78622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:09.361381 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0e36-926f-481c-bad1-3dde61a78622 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:09.362215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0e36-926f-481c-bad1-3dde61a78622 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:09.362948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c0e36-926f-481c-bad1-3dde61a78622 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:09.386608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c0e36-926f-481c-bad1-3dde61a78622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:09.398378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-596c0e36-926f-481c-bad1-3dde61a78622 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:09.398378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:09.398378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:09.398378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:09.398378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:09.398378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:09.398378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:09.398378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:09.398378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:09.398378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:09.398378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:09.398378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:09.398378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:09.398378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:09.398378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:09.398378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:09.400129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:09.400129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:09.400129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:09.400129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:09.400129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:09.400129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:09.400129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:09.400129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:09.400129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:09.400129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:09.400129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:09.400129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:09.400129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:09.400129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:09.400129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:09.400129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:09.401656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:09.401656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:09.401656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:09.401656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:09.401656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:09.401656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:09.401656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:09.401656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:09.401656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:09.401656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:09.401656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:09.401656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:09.406490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 239/488] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:09.416555 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c0e36-926f-481c-bad1-3dde61a78622 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:09.588203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c0714-ba85-4bc0-848c-0a1d40b3330a None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:09.618221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:09.642832 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c0e36-926f-481c-bad1-3dde61a78622 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:09.670273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785ace0-19c7-44ca-8aec-112f81ecbb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:09.672270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785ace0-19c7-44ca-8aec-112f81ecbb27 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:09.673886 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785ace0-19c7-44ca-8aec-112f81ecbb27 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:09.675261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785ace0-19c7-44ca-8aec-112f81ecbb27 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:09.677601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785ace0-19c7-44ca-8aec-112f81ecbb27 None admin] RBAC: Adding query filter params (name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:09.678671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785ace0-19c7-44ca-8aec-112f81ecbb27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:09.681944 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785ace0-19c7-44ca-8aec-112f81ecbb27 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:09.702381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 250/489] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:09:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 496 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:09.705862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785ace0-19c7-44ca-8aec-112f81ecbb27 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:09.715502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:09.725688 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cc560-e0d0-4a05-8286-1c8465b8d569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:09.727815 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cc560-e0d0-4a05-8286-1c8465b8d569 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:09.728437 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cc560-e0d0-4a05-8286-1c8465b8d569 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:09.728937 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cc560-e0d0-4a05-8286-1c8465b8d569 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:09.730440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cc560-e0d0-4a05-8286-1c8465b8d569 None admin] RBAC: Adding query filter params (name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:09.731142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cc560-e0d0-4a05-8286-1c8465b8d569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:09.734611 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cc560-e0d0-4a05-8286-1c8465b8d569 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:09.763390 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 240/490] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:09:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:09:09.765292 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cc560-e0d0-4a05-8286-1c8465b8d569 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:09.937333 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785ace0-19c7-44ca-8aec-112f81ecbb27 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:09.976796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cc560-e0d0-4a05-8286-1c8465b8d569 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:10.018427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c05d6-a824-49c0-b727-81c3f76e3484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:10.019586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c05d6-a824-49c0-b727-81c3f76e3484 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:10.020071 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c05d6-a824-49c0-b727-81c3f76e3484 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:10.020553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c05d6-a824-49c0-b727-81c3f76e3484 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:10.042367 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c05d6-a824-49c0-b727-81c3f76e3484 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:10.047002 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36cee0-c942-4d56-a140-787f53ad0901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:10.048323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36cee0-c942-4d56-a140-787f53ad0901 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:10.048848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36cee0-c942-4d56-a140-787f53ad0901 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:10.049348 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36cee0-c942-4d56-a140-787f53ad0901 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:10.053210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-829c05d6-a824-49c0-b727-81c3f76e3484 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:10.053210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:10.053210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:10.053210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:10.053210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:10.053210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:10.053210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:10.053210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:10.053210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:10.053210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:10.053210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:10.053210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:10.053210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:10.053210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:10.053210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:10.053210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:10.055045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:10.055045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:10.055045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:10.055045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:10.055045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:10.055045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:10.055045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:10.055045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:10.055045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:10.055045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:10.055045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:10.055045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:10.055045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:10.055045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:10.055045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:10.055045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:10.056672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:10.056672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:10.056672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:10.056672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:10.056672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:10.056672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:10.056672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:10.056672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:10.056672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:10.056672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:10.056672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:10.056672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:10.056672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 251/491] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:10.059185 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c05d6-a824-49c0-b727-81c3f76e3484 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:10.073983 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36cee0-c942-4d56-a140-787f53ad0901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:10.077020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36cee0-c942-4d56-a140-787f53ad0901 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:10.108355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-4f36cee0-c942-4d56-a140-787f53ad0901 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:10.108355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:10.108355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:10.108355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:10.108355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:10.108355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:10.108355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:10.108355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:10.108355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:10.108355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:10.108355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:10.108355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:10.108355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:10.108355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:10.108355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:10.108355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:10.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:10.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:10.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:10.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:10.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:10.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:10.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:10.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:10.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:10.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:10.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:10.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:10.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:10.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:10.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:10.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:10.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:10.112505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:10.112505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:10.112505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:10.112505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 241/492] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:10.112941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfa75a-dceb-4e2f-90e5-1f7539dcb967 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:10.113834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfa75a-dceb-4e2f-90e5-1f7539dcb967 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:10.114093 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfa75a-dceb-4e2f-90e5-1f7539dcb967 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:10.117246 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 242/493] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:10.118776 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfa75a-dceb-4e2f-90e5-1f7539dcb967 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:10.125771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:10.199695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:10.288136 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c05d6-a824-49c0-b727-81c3f76e3484 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:10.343325 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfa75a-dceb-4e2f-90e5-1f7539dcb967 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:10.359054 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21e678-c6ea-4db9-b007-c2eb1f5191ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:10.360136 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21e678-c6ea-4db9-b007-c2eb1f5191ee None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:10.360702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21e678-c6ea-4db9-b007-c2eb1f5191ee None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:10.361172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21e678-c6ea-4db9-b007-c2eb1f5191ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:10.383808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21e678-c6ea-4db9-b007-c2eb1f5191ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:10.386532 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21e678-c6ea-4db9-b007-c2eb1f5191ee None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:10.413759 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df411875-c050-4f1f-b453-7d0df3477bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:10.414764 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df411875-c050-4f1f-b453-7d0df3477bde None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:10.415340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df411875-c050-4f1f-b453-7d0df3477bde None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:10.415751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df411875-c050-4f1f-b453-7d0df3477bde None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:10.416956 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df411875-c050-4f1f-b453-7d0df3477bde None admin] RBAC: Adding query filter params (name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:10.417500 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df411875-c050-4f1f-b453-7d0df3477bde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:10.418325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-fa21e678-c6ea-4db9-b007-c2eb1f5191ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:10.418325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:10.418325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:10.418325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:10.418325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:10.418325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:10.418325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:10.418325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:10.418325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:10.418325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:10.418325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:10.418325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:10.418325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:10.418325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:10.418325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:10.418325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:10.420447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:10.420447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:10.420447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:10.420447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:10.420447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:10.420447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:10.420447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:10.420447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:10.420447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:10.420447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:10.420447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:10.420447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:10.420447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:10.420447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:10.420447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:10.420447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:10.420447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:10.423625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:10.423625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:10.423625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:10.424306 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df411875-c050-4f1f-b453-7d0df3477bde None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:10.427470 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 252/494] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:10.428805 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21e678-c6ea-4db9-b007-c2eb1f5191ee None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:10.447750 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 243/495] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:09:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:10.449605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0119c-fa95-48bd-8d9c-20ae4f79491f None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:10.450937 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0119c-fa95-48bd-8d9c-20ae4f79491f None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:10.450937 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0119c-fa95-48bd-8d9c-20ae4f79491f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:10.486087 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-47f0119c-fa95-48bd-8d9c-20ae4f79491f None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:10.488221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-47f0119c-fa95-48bd-8d9c-20ae4f79491f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:10.622884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:10.638805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 244/496] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:10.640420 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0119c-fa95-48bd-8d9c-20ae4f79491f None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:10.680134 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21e678-c6ea-4db9-b007-c2eb1f5191ee None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:10.717084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:10.763392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6ad52-0872-4244-a885-0275c4f1a74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:10.764582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6ad52-0872-4244-a885-0275c4f1a74a None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:10.765092 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6ad52-0872-4244-a885-0275c4f1a74a None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:10.765597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6ad52-0872-4244-a885-0275c4f1a74a None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:10.767416 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6ad52-0872-4244-a885-0275c4f1a74a None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:10.768068 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6ad52-0872-4244-a885-0275c4f1a74a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:10.771005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6ad52-0872-4244-a885-0275c4f1a74a None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:10.801414 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 253/497] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:10.808866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6ad52-0872-4244-a885-0275c4f1a74a None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:10.893269 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0119c-fa95-48bd-8d9c-20ae4f79491f None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:10.964110 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d649600-b6e3-4a69-a5b1-6a07cef8fd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:10.965203 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d649600-b6e3-4a69-a5b1-6a07cef8fd6a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:10.965717 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d649600-b6e3-4a69-a5b1-6a07cef8fd6a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:10.966223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d649600-b6e3-4a69-a5b1-6a07cef8fd6a None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:10.967639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d649600-b6e3-4a69-a5b1-6a07cef8fd6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:10.968118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d649600-b6e3-4a69-a5b1-6a07cef8fd6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:10.971723 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d649600-b6e3-4a69-a5b1-6a07cef8fd6a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:10.994527 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 245/498] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:10.996339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d649600-b6e3-4a69-a5b1-6a07cef8fd6a None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:11.042769 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6ad52-0872-4244-a885-0275c4f1a74a None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:11.105416 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a7cbc-fb28-4153-b0e2-d17f5f58abcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:11.106177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a7cbc-fb28-4153-b0e2-d17f5f58abcb None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:11.106532 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a7cbc-fb28-4153-b0e2-d17f5f58abcb None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:11.106802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a7cbc-fb28-4153-b0e2-d17f5f58abcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:11.107149 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:11.127124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a7cbc-fb28-4153-b0e2-d17f5f58abcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:11.130063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a7cbc-fb28-4153-b0e2-d17f5f58abcb None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:11.163368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-3f2a7cbc-fb28-4153-b0e2-d17f5f58abcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:11.163368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:11.163368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:11.163368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:11.163368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:11.163368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:11.163368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:11.163368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:11.163368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:11.163368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:11.163368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:11.163368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:11.163368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:11.163368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:11.163368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:11.163368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:11.165566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:11.165566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:11.165566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:11.165566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:11.165566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:11.165566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:11.165566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:11.165566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:11.165566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:11.165566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:11.165566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:11.165566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:11.165566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:11.165566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:11.165566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:11.165566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:11.165566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:11.167748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:11.167748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:11.167748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:11.167748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 254/499] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:11.167748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474516a9-2bc0-4299-b17f-fd9eb4a6b088 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:11.168460 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474516a9-2bc0-4299-b17f-fd9eb4a6b088 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:11.168598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474516a9-2bc0-4299-b17f-fd9eb4a6b088 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:11.171236 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 255/500] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:11.172666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474516a9-2bc0-4299-b17f-fd9eb4a6b088 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:11.195659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d649600-b6e3-4a69-a5b1-6a07cef8fd6a None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:11.229875 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:11.248099 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1daf9b-f2f1-4cd7-93fe-10e66e69296e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:11.248868 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1daf9b-f2f1-4cd7-93fe-10e66e69296e None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:11.249403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1daf9b-f2f1-4cd7-93fe-10e66e69296e None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:11.249797 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1daf9b-f2f1-4cd7-93fe-10e66e69296e None admin] PATH_INFO: `/v3/users/glance` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:11.267336 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1daf9b-f2f1-4cd7-93fe-10e66e69296e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:11.269805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1daf9b-f2f1-4cd7-93fe-10e66e69296e None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:11.293408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-ed1daf9b-f2f1-4cd7-93fe-10e66e69296e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 13:09:11.293408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:11.293408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:11.293408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:11.293408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:11.293408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:11.293408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:11.293408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:11.293408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:11.293408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:11.293408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:11.293408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:11.293408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:11.293408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:11.293408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:11.293408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:11.295311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:11.297130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:11.297130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:11.297130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:11.297130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:11.297130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:11.297130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:11.297130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:11.297130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:11.297130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 13:09:11.297130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:11.297130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 246/501] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:09:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:11.297130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1daf9b-f2f1-4cd7-93fe-10e66e69296e None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:11.371012 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474516a9-2bc0-4299-b17f-fd9eb4a6b088 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:11.427238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988ada6-5297-4b83-890e-5b27bf2bddc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:11.428558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988ada6-5297-4b83-890e-5b27bf2bddc2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:11.428764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988ada6-5297-4b83-890e-5b27bf2bddc2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:11.429252 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988ada6-5297-4b83-890e-5b27bf2bddc2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:11.443973 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988ada6-5297-4b83-890e-5b27bf2bddc2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:11.446730 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988ada6-5297-4b83-890e-5b27bf2bddc2 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:11.462369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1daf9b-f2f1-4cd7-93fe-10e66e69296e None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:11.473611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-6988ada6-5297-4b83-890e-5b27bf2bddc2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 13:09:11.473611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:11.473611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:11.473611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:11.473611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:11.473611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:11.473611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:11.473611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:11.473611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:11.473611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:11.473611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:11.473611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:11.473611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:11.473611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:11.473611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:11.473611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:11.475574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:11.477181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:11.477181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:11.477181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:11.477181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:11.477181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:11.477181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:11.477181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:11.477181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:11.477181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 13:09:11.477181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:11.477181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 256/502] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:09:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:11.478304 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988ada6-5297-4b83-890e-5b27bf2bddc2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:11.491193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:11.525895 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5570c2-e1c2-451b-a7fb-bf29ec7d1b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:11.526958 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5570c2-e1c2-451b-a7fb-bf29ec7d1b6a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:11.527487 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5570c2-e1c2-451b-a7fb-bf29ec7d1b6a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:11.527965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5570c2-e1c2-451b-a7fb-bf29ec7d1b6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:11.549313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5570c2-e1c2-451b-a7fb-bf29ec7d1b6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:11.560459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-9e5570c2-e1c2-451b-a7fb-bf29ec7d1b6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:11.560459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:11.560459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:11.560459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:11.560459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:11.560459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:11.560459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:11.560459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:11.560459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:11.560459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:11.560459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:11.560459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:11.560459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:11.560459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:11.560459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:11.560459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:11.562272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:11.562272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:11.562272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:11.562272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:11.562272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:11.562272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:11.562272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:11.562272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:11.562272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:11.562272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:11.562272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:11.562272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:11.562272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:11.562272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:11.562272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:11.562272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:11.565070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:11.565070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:11.565070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:11.565070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:11.565070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:11.565070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:11.565070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:11.565070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:11.565070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:11.565070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:11.565070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:11.565070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:11.565070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 247/503] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:11.566336 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2e5b2-e62b-45df-a64a-36d72aa870b2 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:11.567640 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2e5b2-e62b-45df-a64a-36d72aa870b2 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:11.567640 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2e5b2-e62b-45df-a64a-36d72aa870b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:11.600857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-e6c2e5b2-e62b-45df-a64a-36d72aa870b2 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:11.603331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e6c2e5b2-e62b-45df-a64a-36d72aa870b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:11.644639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:11.667957 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988ada6-5297-4b83-890e-5b27bf2bddc2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:11.730061 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f0961-a5c4-4151-adfa-3a5f508c3bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:11.731217 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f0961-a5c4-4151-adfa-3a5f508c3bb8 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:11.731657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f0961-a5c4-4151-adfa-3a5f508c3bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:11.732152 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f0961-a5c4-4151-adfa-3a5f508c3bb8 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:11.732935 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 248/504] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:11.733369 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f0961-a5c4-4151-adfa-3a5f508c3bb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:11.733925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f0961-a5c4-4151-adfa-3a5f508c3bb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:11.734526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2e5b2-e62b-45df-a64a-36d72aa870b2 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:11.737201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f0961-a5c4-4151-adfa-3a5f508c3bb8 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:11.760428 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 257/505] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:11.762599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f0961-a5c4-4151-adfa-3a5f508c3bb8 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:11.957959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2e5b2-e62b-45df-a64a-36d72aa870b2 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:11.963789 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:11.998605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f0961-a5c4-4151-adfa-3a5f508c3bb8 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:12.024596 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcea8a3-3c5d-4adb-9591-1e8db85f4bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:12.025387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcea8a3-3c5d-4adb-9591-1e8db85f4bdb None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:12.025815 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcea8a3-3c5d-4adb-9591-1e8db85f4bdb None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:12.026136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcea8a3-3c5d-4adb-9591-1e8db85f4bdb None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:12.027200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcea8a3-3c5d-4adb-9591-1e8db85f4bdb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:12.027513 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcea8a3-3c5d-4adb-9591-1e8db85f4bdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:12.029316 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcea8a3-3c5d-4adb-9591-1e8db85f4bdb None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:12.071399 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b619c-9698-43a6-8aea-8c4bbbb514e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:12.072339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b619c-9698-43a6-8aea-8c4bbbb514e7 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:12.072767 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b619c-9698-43a6-8aea-8c4bbbb514e7 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:12.073193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b619c-9698-43a6-8aea-8c4bbbb514e7 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:12.074454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b619c-9698-43a6-8aea-8c4bbbb514e7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:12.074931 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b619c-9698-43a6-8aea-8c4bbbb514e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:12.077230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b619c-9698-43a6-8aea-8c4bbbb514e7 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:12.113297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 249/506] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Sep 7 13:09:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:12.113297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcea8a3-3c5d-4adb-9591-1e8db85f4bdb None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:12.162782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 258/507] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Sep 7 13:09:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:12.162782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b619c-9698-43a6-8aea-8c4bbbb514e7 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:12.169217 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:12.291946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcea8a3-3c5d-4adb-9591-1e8db85f4bdb None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:12.355387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61ab71-4f4b-4a79-a5de-430a4b806ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:12.356398 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61ab71-4f4b-4a79-a5de-430a4b806ab2 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:12.356894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61ab71-4f4b-4a79-a5de-430a4b806ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:12.357393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61ab71-4f4b-4a79-a5de-430a4b806ab2 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:12.358083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b619c-9698-43a6-8aea-8c4bbbb514e7 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:12.358654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61ab71-4f4b-4a79-a5de-430a4b806ab2 None admin] RBAC: Adding query filter params (name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:12.359244 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61ab71-4f4b-4a79-a5de-430a4b806ab2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:12.362719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61ab71-4f4b-4a79-a5de-430a4b806ab2 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:12.382706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 250/508] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:09:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:12.384484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61ab71-4f4b-4a79-a5de-430a4b806ab2 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:12.412344 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0ce38-a8af-46c8-b670-7515e046b566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:12.413001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0ce38-a8af-46c8-b670-7515e046b566 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:12.413317 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0ce38-a8af-46c8-b670-7515e046b566 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:12.413743 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0ce38-a8af-46c8-b670-7515e046b566 None admin] PATH_INFO: `/v3/users/nova` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:12.425087 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0ce38-a8af-46c8-b670-7515e046b566 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:12.426814 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0ce38-a8af-46c8-b670-7515e046b566 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:12.435508 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:12.446392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-abe0ce38-a8af-46c8-b670-7515e046b566 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:09:12.446392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:12.446392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:12.446392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:12.446392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:12.446392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:12.446392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:12.446392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:12.446392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:12.446392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:12.446392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:12.446392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:12.446392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:12.446392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:12.446392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:12.446392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:12.460030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:12.461444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:12.461444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:12.461444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:12.461444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:12.461444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:12.461444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:12.461444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:12.461444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:12.461444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:09:12.461444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:12.461444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 259/509] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 7 13:09:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:12.461444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0ce38-a8af-46c8-b670-7515e046b566 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:12.578021 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:12.578499 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61ab71-4f4b-4a79-a5de-430a4b806ab2 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:12.643385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c0085-5f99-493d-a6cc-a5e01714e6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:12.644454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c0085-5f99-493d-a6cc-a5e01714e6f6 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:12.644900 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c0085-5f99-493d-a6cc-a5e01714e6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:12.645379 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c0085-5f99-493d-a6cc-a5e01714e6f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:12.655166 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0ce38-a8af-46c8-b670-7515e046b566 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:12.720274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa186de8-8df4-40df-aad1-f1d72ee6eea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:12.721499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa186de8-8df4-40df-aad1-f1d72ee6eea8 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:12.721499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa186de8-8df4-40df-aad1-f1d72ee6eea8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:12.722173 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa186de8-8df4-40df-aad1-f1d72ee6eea8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:12.764722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c0085-5f99-493d-a6cc-a5e01714e6f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:12.768112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c0085-5f99-493d-a6cc-a5e01714e6f6 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:12.816845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:12.847999 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa186de8-8df4-40df-aad1-f1d72ee6eea8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:12.850408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa186de8-8df4-40df-aad1-f1d72ee6eea8 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:12.943923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 251/510] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:09:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:09:12.944962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c0085-5f99-493d-a6cc-a5e01714e6f6 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:12.960150 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:13.031557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 260/511] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:09:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:09:13.033689 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa186de8-8df4-40df-aad1-f1d72ee6eea8 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:13.122663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c0085-5f99-493d-a6cc-a5e01714e6f6 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:13.163262 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:13.181267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3c188-da7c-43cd-9b3d-88b98553f22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:13.182395 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3c188-da7c-43cd-9b3d-88b98553f22f None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:13.182898 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3c188-da7c-43cd-9b3d-88b98553f22f None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:13.183443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3c188-da7c-43cd-9b3d-88b98553f22f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:13.205861 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3c188-da7c-43cd-9b3d-88b98553f22f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:13.207245 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa186de8-8df4-40df-aad1-f1d72ee6eea8 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:13.208664 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3c188-da7c-43cd-9b3d-88b98553f22f None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:13.227920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-95d3c188-da7c-43cd-9b3d-88b98553f22f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:13.227920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:13.227920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:13.227920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:13.227920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:13.227920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:13.227920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:13.227920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:13.227920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:13.227920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:13.227920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:13.227920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:13.227920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:13.227920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:13.227920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:13.227920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:13.229447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:13.229447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:13.229447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:13.229447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:13.229447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:13.229447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:13.229447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:13.229447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:13.229447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:13.229447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:13.229447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:13.229447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:13.229447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:13.229447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:13.229447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:13.229447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:13.229447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:13.231769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:13.231769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:13.231769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:13.231769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 252/512] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:13.231769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3c188-da7c-43cd-9b3d-88b98553f22f None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:13.276975 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a172f6a-80c7-4f9b-8d22-f1245de46bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:13.278040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a172f6a-80c7-4f9b-8d22-f1245de46bf1 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:13.278528 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a172f6a-80c7-4f9b-8d22-f1245de46bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:13.279068 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a172f6a-80c7-4f9b-8d22-f1245de46bf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:13.302631 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a172f6a-80c7-4f9b-8d22-f1245de46bf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:13.305036 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a172f6a-80c7-4f9b-8d22-f1245de46bf1 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:13.325108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-4a172f6a-80c7-4f9b-8d22-f1245de46bf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:13.325108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:13.325108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:13.325108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:13.325108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:13.325108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:13.325108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:13.325108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:13.325108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:13.325108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:13.325108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:13.325108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:13.325108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:13.325108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:13.325108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:13.325108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:13.326104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:13.326104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:13.326104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:13.326104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:13.326104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:13.326104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:13.326104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:13.326104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:13.326104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:13.326104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:13.326104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:13.326104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:13.326104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:13.326104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:13.326104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:13.326104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:13.326104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:13.326996 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:13.326996 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:13.326996 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:13.326996 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 261/513] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:13.327357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a172f6a-80c7-4f9b-8d22-f1245de46bf1 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:13.350393 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:13.404733 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3c188-da7c-43cd-9b3d-88b98553f22f None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:13.451399 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a680170-9672-410c-a6e1-f68d95b619f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:13.452026 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a680170-9672-410c-a6e1-f68d95b619f8 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:13.452383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a680170-9672-410c-a6e1-f68d95b619f8 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:13.452661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a680170-9672-410c-a6e1-f68d95b619f8 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:13.453433 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a680170-9672-410c-a6e1-f68d95b619f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:13.453739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a680170-9672-410c-a6e1-f68d95b619f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:13.456310 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a680170-9672-410c-a6e1-f68d95b619f8 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:13.479063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a172f6a-80c7-4f9b-8d22-f1245de46bf1 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:13.486391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 253/514] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 7 13:09:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:13.487630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a680170-9672-410c-a6e1-f68d95b619f8 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:13.529240 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3f86e-9f38-480c-a358-96d61ee8eaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:13.530095 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3f86e-9f38-480c-a358-96d61ee8eaa3 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:13.530381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3f86e-9f38-480c-a358-96d61ee8eaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:13.530675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3f86e-9f38-480c-a358-96d61ee8eaa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:13.542183 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:13.543403 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d3f86e-9f38-480c-a358-96d61ee8eaa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:13.544979 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d3f86e-9f38-480c-a358-96d61ee8eaa3 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:13.563974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-f7d3f86e-9f38-480c-a358-96d61ee8eaa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:13.563974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:13.563974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:13.563974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:13.563974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:13.563974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:13.563974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:13.563974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:13.563974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:13.563974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:13.563974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:13.563974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:13.563974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:13.563974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:13.563974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:13.563974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:13.565727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:13.565727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:13.565727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:13.565727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:13.565727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:13.565727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:13.565727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:13.565727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:13.565727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:13.565727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:13.565727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:13.565727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:13.565727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:13.565727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:13.565727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:13.565727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:13.565727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:13.568219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:13.568219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:13.568219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:13.568219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 262/515] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:13.568219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3f86e-9f38-480c-a358-96d61ee8eaa3 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:13.662803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a680170-9672-410c-a6e1-f68d95b619f8 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:13.675862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:13.717776 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391013c-a3fa-4a19-944e-c5717448595a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:13.718673 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391013c-a3fa-4a19-944e-c5717448595a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:13.719097 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391013c-a3fa-4a19-944e-c5717448595a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:13.719455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391013c-a3fa-4a19-944e-c5717448595a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:13.731374 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3f86e-9f38-480c-a358-96d61ee8eaa3 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:13.732948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391013c-a3fa-4a19-944e-c5717448595a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:13.734796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391013c-a3fa-4a19-944e-c5717448595a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:13.754627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-3391013c-a3fa-4a19-944e-c5717448595a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:13.754627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:13.754627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:13.754627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:13.754627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:13.754627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:13.754627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:13.754627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:13.754627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:13.754627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:13.754627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:13.754627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:13.754627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:13.754627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:13.754627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:13.754627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:13.756639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:13.756639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:13.756639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:13.756639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:13.756639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:13.756639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:13.756639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:13.756639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:13.756639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:13.756639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:13.756639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:13.756639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:13.756639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:13.756639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:13.756639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:13.756639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:13.756639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:13.758336 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:13.758336 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:13.758336 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:13.758336 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 254/516] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:13.758336 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391013c-a3fa-4a19-944e-c5717448595a None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:13.794644 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5feee4a-5efe-4e0a-8ff9-e78dbb8956d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:13.795514 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5feee4a-5efe-4e0a-8ff9-e78dbb8956d9 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:13.796081 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5feee4a-5efe-4e0a-8ff9-e78dbb8956d9 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:13.796595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5feee4a-5efe-4e0a-8ff9-e78dbb8956d9 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:13.797979 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5feee4a-5efe-4e0a-8ff9-e78dbb8956d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:13.798517 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5feee4a-5efe-4e0a-8ff9-e78dbb8956d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:13.801232 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5feee4a-5efe-4e0a-8ff9-e78dbb8956d9 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:13.826319 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 263/517] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:13.827496 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5feee4a-5efe-4e0a-8ff9-e78dbb8956d9 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:13.915486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:13.935980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391013c-a3fa-4a19-944e-c5717448595a None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:13.995808 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85337e2-6689-4445-b665-92904e74647b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:13.996776 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85337e2-6689-4445-b665-92904e74647b None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:13.997188 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85337e2-6689-4445-b665-92904e74647b None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:13.997584 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85337e2-6689-4445-b665-92904e74647b None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:13.998627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85337e2-6689-4445-b665-92904e74647b None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:13.999152 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85337e2-6689-4445-b665-92904e74647b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:14.001972 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85337e2-6689-4445-b665-92904e74647b None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:14.005238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5feee4a-5efe-4e0a-8ff9-e78dbb8956d9 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:14.021828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 255/518] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:14.024055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85337e2-6689-4445-b665-92904e74647b None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:14.055312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:14.067147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3406fef-41d4-42df-be3d-40c5791ea03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:14.068049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3406fef-41d4-42df-be3d-40c5791ea03f None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:14.068530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3406fef-41d4-42df-be3d-40c5791ea03f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:14.069113 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3406fef-41d4-42df-be3d-40c5791ea03f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:14.193845 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3406fef-41d4-42df-be3d-40c5791ea03f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:14.196991 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3406fef-41d4-42df-be3d-40c5791ea03f None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:14.199214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85337e2-6689-4445-b665-92904e74647b None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:14.261215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7bb9c-79b5-4a18-9732-5f8fa7347ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:14.262065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7bb9c-79b5-4a18-9732-5f8fa7347ec9 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:14.262366 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7bb9c-79b5-4a18-9732-5f8fa7347ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:14.262747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7bb9c-79b5-4a18-9732-5f8fa7347ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:14.263557 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7bb9c-79b5-4a18-9732-5f8fa7347ec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:14.263957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7bb9c-79b5-4a18-9732-5f8fa7347ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:14.265812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7bb9c-79b5-4a18-9732-5f8fa7347ec9 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:14.284130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 256/519] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:14.285229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7bb9c-79b5-4a18-9732-5f8fa7347ec9 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:14.304788 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:14.388549 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 264/520] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:09:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:09:14.391472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3406fef-41d4-42df-be3d-40c5791ea03f None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:14.427750 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:14.449608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7bb9c-79b5-4a18-9732-5f8fa7347ec9 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:14.500710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7ab17-00a6-41ae-96ce-5d13f63ebb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:14.502418 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7ab17-00a6-41ae-96ce-5d13f63ebb8f None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:14.502910 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7ab17-00a6-41ae-96ce-5d13f63ebb8f None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:14.503484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7ab17-00a6-41ae-96ce-5d13f63ebb8f None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:14.504926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7ab17-00a6-41ae-96ce-5d13f63ebb8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:14.505480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7ab17-00a6-41ae-96ce-5d13f63ebb8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:14.508231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7ab17-00a6-41ae-96ce-5d13f63ebb8f None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:14.529082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 257/521] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:14.530006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7ab17-00a6-41ae-96ce-5d13f63ebb8f None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:14.558678 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3406fef-41d4-42df-be3d-40c5791ea03f None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:14.644649 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc7393-1a70-4fc4-9f73-8346dde3fce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:14.645825 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc7393-1a70-4fc4-9f73-8346dde3fce4 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:14.646333 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc7393-1a70-4fc4-9f73-8346dde3fce4 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:14.646828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc7393-1a70-4fc4-9f73-8346dde3fce4 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:14.648714 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc7393-1a70-4fc4-9f73-8346dde3fce4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:14.651714 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc7393-1a70-4fc4-9f73-8346dde3fce4 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:14.661720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:14.739806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7ab17-00a6-41ae-96ce-5d13f63ebb8f None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:14.814888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74eb968-1a8e-421e-8257-5ec6981e6d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:14.815999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74eb968-1a8e-421e-8257-5ec6981e6d7e None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:14.816445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74eb968-1a8e-421e-8257-5ec6981e6d7e None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:14.816900 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74eb968-1a8e-421e-8257-5ec6981e6d7e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:14.817987 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 265/522] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:09:14 2022] POST /identity/v3/users => generated 247 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:09:14.818929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc7393-1a70-4fc4-9f73-8346dde3fce4 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:14.833754 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74eb968-1a8e-421e-8257-5ec6981e6d7e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:14.836610 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74eb968-1a8e-421e-8257-5ec6981e6d7e None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:14.862159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-d74eb968-1a8e-421e-8257-5ec6981e6d7e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 13:09:14.862159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:14.862159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:14.862159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:14.862159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:14.862159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:14.862159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:14.862159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:14.862159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:14.862159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:14.862159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:14.862159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:14.862159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:14.862159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:14.862159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:14.862159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:14.864214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:14.866113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:14.866113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:14.866113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:14.866113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:14.866113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:14.866113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:14.866113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:14.866113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:14.866113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 13:09:14.866113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:14.866113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 258/523] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:14.866113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74eb968-1a8e-421e-8257-5ec6981e6d7e None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:14.989190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:15.024364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc7393-1a70-4fc4-9f73-8346dde3fce4 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:15.055500 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74eb968-1a8e-421e-8257-5ec6981e6d7e None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:15.097126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cd855-0625-4654-aa29-9adfaef4aef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:15.097952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cd855-0625-4654-aa29-9adfaef4aef6 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:15.099085 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cd855-0625-4654-aa29-9adfaef4aef6 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:15.099681 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cd855-0625-4654-aa29-9adfaef4aef6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:15.112984 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cd855-0625-4654-aa29-9adfaef4aef6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:15.113249 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:15.116075 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cd855-0625-4654-aa29-9adfaef4aef6 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:15.133855 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ce663-2cf3-433f-8c54-547c23fe7761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:15.135244 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ce663-2cf3-433f-8c54-547c23fe7761 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:15.135816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ce663-2cf3-433f-8c54-547c23fe7761 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:15.136432 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ce663-2cf3-433f-8c54-547c23fe7761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:15.142519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-889cd855-0625-4654-aa29-9adfaef4aef6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:15.142519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:15.142519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:15.142519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:15.142519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:15.142519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:15.142519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:15.142519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:15.142519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:15.142519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:15.142519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:15.142519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:15.142519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:15.142519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:15.142519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:15.144481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:15.144481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:15.144481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:15.144481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:15.144481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:15.144481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:15.144481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:15.144481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:15.144481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:15.144481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:15.144481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:15.144481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:15.144481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:15.144481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:15.144481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:15.144481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:15.144481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:15.146145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:15.146145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:15.146145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:15.146145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:15.146145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:15.146145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:15.146145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:15.146145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:15.146145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 266/524] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:15.147136 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cd855-0625-4654-aa29-9adfaef4aef6 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:15.162056 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ce663-2cf3-433f-8c54-547c23fe7761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:15.164921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ce663-2cf3-433f-8c54-547c23fe7761 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:15.194176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-448ce663-2cf3-433f-8c54-547c23fe7761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:15.194176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:15.194176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:15.194176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:15.194176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:15.194176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:15.194176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:15.194176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:15.194176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:15.194176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:15.194176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:15.194176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:15.194176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:15.194176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:15.194176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:15.194176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:15.196251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:15.196251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:15.196251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:15.196251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:15.196251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:15.196251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:15.196251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:15.196251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:15.196251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:15.196251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:15.196251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:15.196251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:15.196251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:15.196251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:15.196251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:15.196251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:15.196251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:15.198150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:15.198150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:15.198150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:15.198150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 259/525] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:15.198150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ce663-2cf3-433f-8c54-547c23fe7761 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:15.339069 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cd855-0625-4654-aa29-9adfaef4aef6 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:15.418611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fdfa47-dbfb-4ec0-b70c-8c4f241b9d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:15.419740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdfa47-dbfb-4ec0-b70c-8c4f241b9d8c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:15.420418 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdfa47-dbfb-4ec0-b70c-8c4f241b9d8c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:15.420993 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdfa47-dbfb-4ec0-b70c-8c4f241b9d8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:15.422632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ce663-2cf3-433f-8c54-547c23fe7761 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:15.433798 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fdfa47-dbfb-4ec0-b70c-8c4f241b9d8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:15.436879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fdfa47-dbfb-4ec0-b70c-8c4f241b9d8c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:15.461697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-29fdfa47-dbfb-4ec0-b70c-8c4f241b9d8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:15.461697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:15.461697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:15.461697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:15.461697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:15.461697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:15.461697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:15.461697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:15.461697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:15.461697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:15.461697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:15.461697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:15.461697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:15.461697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:15.461697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:15.463335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:15.463335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:15.463335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:15.463335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:15.463335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:15.463335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:15.463335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:15.463335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:15.463335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:15.463335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:15.463335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:15.463335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:15.463335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:15.463335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:15.463335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:15.463335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:15.463335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:15.465067 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:15.465067 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:15.465067 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:15.465067 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:15.465067 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:15.465067 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:15.465067 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:15.465067 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:15.467342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 267/526] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:15.468373 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fdfa47-dbfb-4ec0-b70c-8c4f241b9d8c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:15.473506 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:15.498369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89767d21-3915-417d-8a4d-1149cd369c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:15.499855 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89767d21-3915-417d-8a4d-1149cd369c2a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:15.500341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89767d21-3915-417d-8a4d-1149cd369c2a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:15.500894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89767d21-3915-417d-8a4d-1149cd369c2a None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:15.502233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89767d21-3915-417d-8a4d-1149cd369c2a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:15.502766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89767d21-3915-417d-8a4d-1149cd369c2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:15.505993 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89767d21-3915-417d-8a4d-1149cd369c2a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:15.539663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 260/527] 10.222.0.62 () {56 vars in 1139 bytes} [Wed Sep 7 13:09:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:15.540821 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89767d21-3915-417d-8a4d-1149cd369c2a None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:15.607999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:15.655637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fdfa47-dbfb-4ec0-b70c-8c4f241b9d8c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:15.724138 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28312ddf-057e-442a-99e4-184c84a2c6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:15.725199 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28312ddf-057e-442a-99e4-184c84a2c6c1 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:15.725787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28312ddf-057e-442a-99e4-184c84a2c6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:15.726268 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28312ddf-057e-442a-99e4-184c84a2c6c1 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:15.727599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28312ddf-057e-442a-99e4-184c84a2c6c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:15.728148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28312ddf-057e-442a-99e4-184c84a2c6c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:15.730854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28312ddf-057e-442a-99e4-184c84a2c6c1 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:15.741103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89767d21-3915-417d-8a4d-1149cd369c2a None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:15.768184 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 268/528] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:15.770189 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28312ddf-057e-442a-99e4-184c84a2c6c1 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:15.825370 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc5e62-5575-4ae4-b7c8-dd5bc29ddcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:15.826590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc5e62-5575-4ae4-b7c8-dd5bc29ddcd5 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:15.826903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc5e62-5575-4ae4-b7c8-dd5bc29ddcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:15.827418 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc5e62-5575-4ae4-b7c8-dd5bc29ddcd5 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:15.828631 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc5e62-5575-4ae4-b7c8-dd5bc29ddcd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:15.829227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc5e62-5575-4ae4-b7c8-dd5bc29ddcd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:15.832126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc5e62-5575-4ae4-b7c8-dd5bc29ddcd5 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:15.857772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 261/529] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:15.859200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc5e62-5575-4ae4-b7c8-dd5bc29ddcd5 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:15.926650 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:16.012673 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28312ddf-057e-442a-99e4-184c84a2c6c1 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:16.076042 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:16.088116 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd99fc0-cf95-416c-a887-4e935c44b180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:16.088598 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc5e62-5575-4ae4-b7c8-dd5bc29ddcd5 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:16.089710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd99fc0-cf95-416c-a887-4e935c44b180 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:16.090316 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd99fc0-cf95-416c-a887-4e935c44b180 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:16.090893 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd99fc0-cf95-416c-a887-4e935c44b180 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:16.097434 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd99fc0-cf95-416c-a887-4e935c44b180 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:16.097434 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd99fc0-cf95-416c-a887-4e935c44b180 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:16.097434 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd99fc0-cf95-416c-a887-4e935c44b180 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:16.138136 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 269/530] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:16.138136 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd99fc0-cf95-416c-a887-4e935c44b180 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:16.164207 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854221fc-ad7e-43c9-9e3e-a19b608d0145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:16.165976 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854221fc-ad7e-43c9-9e3e-a19b608d0145 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:16.165976 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854221fc-ad7e-43c9-9e3e-a19b608d0145 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:16.166298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854221fc-ad7e-43c9-9e3e-a19b608d0145 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:16.323115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854221fc-ad7e-43c9-9e3e-a19b608d0145 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:16.326165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854221fc-ad7e-43c9-9e3e-a19b608d0145 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:16.350869 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd99fc0-cf95-416c-a887-4e935c44b180 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:16.406383 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:16.428907 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857e288-9d4e-42d4-ab4d-d80bff04ed8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:16.430138 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857e288-9d4e-42d4-ab4d-d80bff04ed8d None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:16.430719 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857e288-9d4e-42d4-ab4d-d80bff04ed8d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:16.431292 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857e288-9d4e-42d4-ab4d-d80bff04ed8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:16.432685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c857e288-9d4e-42d4-ab4d-d80bff04ed8d None admin] RBAC: Adding query filter params (role.id=743ed2d8e7114975bd9245742a41fe0f, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=d1a48805d95640489d7ce3333dd687b9) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:16.433310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c857e288-9d4e-42d4-ab4d-d80bff04ed8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:16.436343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c857e288-9d4e-42d4-ab4d-d80bff04ed8d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:16.456814 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 270/531] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:09:16 2022] GET /identity/v3/role_assignments?user.id=d1a48805d95640489d7ce3333dd687b9&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=743ed2d8e7114975bd9245742a41fe0f => generated 259 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:16.456814 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857e288-9d4e-42d4-ab4d-d80bff04ed8d None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:16.505839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:16.523405 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 262/532] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:09:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:09:16.525425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854221fc-ad7e-43c9-9e3e-a19b608d0145 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:16.679972 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c857e288-9d4e-42d4-ab4d-d80bff04ed8d None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:16.754732 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854221fc-ad7e-43c9-9e3e-a19b608d0145 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:16.763018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349bfed-daf1-4b37-89c2-28c19731bb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:16.764571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349bfed-daf1-4b37-89c2-28c19731bb91 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:16.765043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349bfed-daf1-4b37-89c2-28c19731bb91 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:16.765521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349bfed-daf1-4b37-89c2-28c19731bb91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:16.789036 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349bfed-daf1-4b37-89c2-28c19731bb91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:16.794837 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349bfed-daf1-4b37-89c2-28c19731bb91 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:16.826712 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bb7c5-47b4-4a4f-b337-5590258dcab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:16.827837 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bb7c5-47b4-4a4f-b337-5590258dcab8 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:16.828330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bb7c5-47b4-4a4f-b337-5590258dcab8 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:16.828837 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6bb7c5-47b4-4a4f-b337-5590258dcab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:16.830843 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bb7c5-47b4-4a4f-b337-5590258dcab8 None admin] RBAC: Adding query filter params (role.id=743ed2d8e7114975bd9245742a41fe0f, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=fc2fa0fa46b842ceb1cac437166b8aea) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:16.831651 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bb7c5-47b4-4a4f-b337-5590258dcab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:16.834406 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6bb7c5-47b4-4a4f-b337-5590258dcab8 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:16.837847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-0349bfed-daf1-4b37-89c2-28c19731bb91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:16.837847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:16.837847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:16.837847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:16.837847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:16.837847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:16.837847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:16.837847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:16.837847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:16.837847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:16.837847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:16.837847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:16.837847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:16.837847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:16.837847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:16.839599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:16.839599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:16.839599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:16.839599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:16.839599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:16.839599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:16.839599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:16.839599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:16.839599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:16.839599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:16.839599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:16.839599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:16.839599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:16.839599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:16.839599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:16.839599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:16.839599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:16.841584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:16.841584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:16.841584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:16.841584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:16.841584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:16.841584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:16.841584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:16.841584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:16.841584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 271/533] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:16.843771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349bfed-daf1-4b37-89c2-28c19731bb91 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:16.858779 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 263/534] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:09:16 2022] GET /identity/v3/role_assignments?user.id=fc2fa0fa46b842ceb1cac437166b8aea&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=743ed2d8e7114975bd9245742a41fe0f => generated 259 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:16.860989 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bb7c5-47b4-4a4f-b337-5590258dcab8 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:16.967634 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:16.996472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:17.061308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349bfed-daf1-4b37-89c2-28c19731bb91 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:17.092399 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6bb7c5-47b4-4a4f-b337-5590258dcab8 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:17.177762 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7962c-ed25-466b-b699-851d0effdc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:17.179381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7962c-ed25-466b-b699-851d0effdc38 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:17.179973 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff86f4-1b09-468a-9682-8c8cba73fbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:17.180536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7962c-ed25-466b-b699-851d0effdc38 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:17.181010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff86f4-1b09-468a-9682-8c8cba73fbf2 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:17.181445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff86f4-1b09-468a-9682-8c8cba73fbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:17.181879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff86f4-1b09-468a-9682-8c8cba73fbf2 None admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:17.182225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7962c-ed25-466b-b699-851d0effdc38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:17.185393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff86f4-1b09-468a-9682-8c8cba73fbf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:17.185393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff86f4-1b09-468a-9682-8c8cba73fbf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:17.187789 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff86f4-1b09-468a-9682-8c8cba73fbf2 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:17.210514 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa7962c-ed25-466b-b699-851d0effdc38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:17.212766 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa7962c-ed25-466b-b699-851d0effdc38 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:17.229046 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 264/535] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:17.240858 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff86f4-1b09-468a-9682-8c8cba73fbf2 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:17.245070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-efa7962c-ed25-466b-b699-851d0effdc38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:17.245070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:17.245070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:17.245070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:17.245070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:17.245070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:17.245070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:17.245070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:17.245070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:17.245070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:17.245070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:17.245070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:17.245070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:17.245070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:17.245070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:17.245070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:17.245994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:17.245994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:17.245994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:17.245994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:17.245994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:17.245994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:17.245994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:17.245994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:17.245994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:17.245994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:17.245994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:17.245994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:17.245994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:17.245994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:17.245994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:17.245994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:17.245994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:17.246902 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:17.246902 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:17.246902 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:17.246902 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 272/536] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:17.256191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7962c-ed25-466b-b699-851d0effdc38 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:17.568073 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff86f4-1b09-468a-9682-8c8cba73fbf2 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:17.578412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:17.591575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7962c-ed25-466b-b699-851d0effdc38 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:17.635575 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:17.664961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e88f4f-9ca3-44e5-ab35-c23cf08dcb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:17.665589 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e88f4f-9ca3-44e5-ab35-c23cf08dcb39 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:17.665870 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e88f4f-9ca3-44e5-ab35-c23cf08dcb39 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:17.666152 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e88f4f-9ca3-44e5-ab35-c23cf08dcb39 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:17.666799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e88f4f-9ca3-44e5-ab35-c23cf08dcb39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:17.667131 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e88f4f-9ca3-44e5-ab35-c23cf08dcb39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:17.668560 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e88f4f-9ca3-44e5-ab35-c23cf08dcb39 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:17.673029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60eb4f-a06e-4499-938c-04ee0df491e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:17.673806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60eb4f-a06e-4499-938c-04ee0df491e3 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:17.674106 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60eb4f-a06e-4499-938c-04ee0df491e3 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:17.674372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60eb4f-a06e-4499-938c-04ee0df491e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:17.675139 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60eb4f-a06e-4499-938c-04ee0df491e3 None admin] RBAC: Adding query filter params (role.id=743ed2d8e7114975bd9245742a41fe0f, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=97c363046db444cbb3d4346066f9ad42) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:17.675567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60eb4f-a06e-4499-938c-04ee0df491e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:17.677721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60eb4f-a06e-4499-938c-04ee0df491e3 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:17.691148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 265/537] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:09:17 2022] GET /identity/v3/role_assignments?user.id=97c363046db444cbb3d4346066f9ad42&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=743ed2d8e7114975bd9245742a41fe0f => generated 259 bytes in 451 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:17.703261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 273/538] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:09:17.709952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60eb4f-a06e-4499-938c-04ee0df491e3 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:17.902661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60eb4f-a06e-4499-938c-04ee0df491e3 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:17.971976 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded553b8-4e8e-496e-bf10-8a3228725cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:17.973140 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded553b8-4e8e-496e-bf10-8a3228725cce None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:17.973635 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded553b8-4e8e-496e-bf10-8a3228725cce None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:17.974118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded553b8-4e8e-496e-bf10-8a3228725cce None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:17.984346 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded553b8-4e8e-496e-bf10-8a3228725cce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:17.987271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded553b8-4e8e-496e-bf10-8a3228725cce None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:18.005205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-ded553b8-4e8e-496e-bf10-8a3228725cce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:18.005205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:18.005205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:18.005205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:18.005205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:18.005205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:18.005205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:18.005205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:18.005205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:18.005205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:18.005205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:18.005205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:18.005205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:18.005205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:18.005205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:18.006186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:18.006186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:18.006186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:18.006186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:18.006186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:18.006186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:18.006186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:18.006186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:18.006186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:18.006186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:18.006186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:18.006186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:18.006186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:18.006186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:18.006186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:18.006186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:18.006186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:18.007538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:18.007538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:18.007538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:18.007538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:18.007538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:18.007538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:18.007538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:18.007538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:18.010039 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 266/539] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:18.020206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e88f4f-9ca3-44e5-ab35-c23cf08dcb39 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:18.280991 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e88f4f-9ca3-44e5-ab35-c23cf08dcb39 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:18.304793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be79c5-20f7-4bff-ad64-8ade7766826c None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:18.305488 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be79c5-20f7-4bff-ad64-8ade7766826c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:18.306015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be79c5-20f7-4bff-ad64-8ade7766826c None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:18.308210 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 267/540] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:09:18.316334 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1161e5-9960-415b-82a0-111669ba8dc5 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:18.316961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1161e5-9960-415b-82a0-111669ba8dc5 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:18.317651 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1161e5-9960-415b-82a0-111669ba8dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:18.336386 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcd827-692a-46e1-ac93-fefd672b274e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:18.337057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcd827-692a-46e1-ac93-fefd672b274e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:18.337343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcd827-692a-46e1-ac93-fefd672b274e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:18.337610 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcd827-692a-46e1-ac93-fefd672b274e None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:18.338530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcd827-692a-46e1-ac93-fefd672b274e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:18.338840 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcd827-692a-46e1-ac93-fefd672b274e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:18.340627 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcd827-692a-46e1-ac93-fefd672b274e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:18.357829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 274/541] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:18.365457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcd827-692a-46e1-ac93-fefd672b274e None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:18.372783 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-1e1161e5-9960-415b-82a0-111669ba8dc5 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:18.374905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-1e1161e5-9960-415b-82a0-111669ba8dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:18.395976 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:18.524907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:18.583899 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcd827-692a-46e1-ac93-fefd672b274e None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:18.610139 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 268/542] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:18.647142 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21068d-5ca0-47ba-8f82-61441dfebf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:18.648362 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21068d-5ca0-47ba-8f82-61441dfebf05 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:18.648994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21068d-5ca0-47ba-8f82-61441dfebf05 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:18.649491 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21068d-5ca0-47ba-8f82-61441dfebf05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:18.650717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e21068d-5ca0-47ba-8f82-61441dfebf05 None admin] RBAC: Adding query filter params (role.id=743ed2d8e7114975bd9245742a41fe0f, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=5acdbdb6f23b4f10bcbab0387ea67ddf) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:18.651336 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e21068d-5ca0-47ba-8f82-61441dfebf05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:18.653528 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e21068d-5ca0-47ba-8f82-61441dfebf05 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:18.662078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 275/543] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:09:18 2022] GET /identity/v3/role_assignments?user.id=5acdbdb6f23b4f10bcbab0387ea67ddf&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=743ed2d8e7114975bd9245742a41fe0f => generated 259 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:09:19.095458 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1161e5-9960-415b-82a0-111669ba8dc5 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:19.273768 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1161e5-9960-415b-82a0-111669ba8dc5 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:19.399857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798f261-9c16-4b60-914f-6dd88932a166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:19.401019 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798f261-9c16-4b60-914f-6dd88932a166 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:19.401653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798f261-9c16-4b60-914f-6dd88932a166 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:19.402150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798f261-9c16-4b60-914f-6dd88932a166 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:19.418512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e798f261-9c16-4b60-914f-6dd88932a166 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:19.425824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-e798f261-9c16-4b60-914f-6dd88932a166 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:19.425824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:19.425824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:19.425824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:19.425824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:19.425824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:19.425824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:19.425824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:19.425824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:19.425824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:19.425824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:19.425824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:19.425824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:19.425824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:19.425824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:19.425824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:19.426890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:19.426890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:19.426890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:19.426890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:19.426890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:19.426890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:19.426890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:19.426890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:19.426890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:19.426890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:19.426890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:19.426890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:19.426890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:19.426890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:19.426890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:19.426890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:19.427736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:19.427736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:19.427736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:19.427736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:19.427736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:19.427736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:19.427736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:19.427736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:19.427736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:19.427736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:19.427736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:19.427736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:19.430879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 269/544] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:19.440975 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21068d-5ca0-47ba-8f82-61441dfebf05 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:19.685888 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:19.732389 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21068d-5ca0-47ba-8f82-61441dfebf05 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:19.811462 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8dec3-b1f9-47d8-b8b2-d7dc83b5df1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:19.812103 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8dec3-b1f9-47d8-b8b2-d7dc83b5df1c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:19.812428 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8dec3-b1f9-47d8-b8b2-d7dc83b5df1c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:19.812700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8dec3-b1f9-47d8-b8b2-d7dc83b5df1c None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:19.814165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8dec3-b1f9-47d8-b8b2-d7dc83b5df1c None admin] RBAC: Adding query filter params (name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:19.814473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8dec3-b1f9-47d8-b8b2-d7dc83b5df1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:19.816026 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8dec3-b1f9-47d8-b8b2-d7dc83b5df1c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:19.831793 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 276/545] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:09:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:19.840340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798f261-9c16-4b60-914f-6dd88932a166 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:19.947285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:20.117755 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798f261-9c16-4b60-914f-6dd88932a166 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:20.195737 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a46c7e-e5d9-4c10-be9e-37f16d9366c9 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:20.196222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a46c7e-e5d9-4c10-be9e-37f16d9366c9 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:20.197467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a46c7e-e5d9-4c10-be9e-37f16d9366c9 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:20.200613 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 277/546] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:20.210724 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd90345-7e16-4c30-910c-dbd93e3aa1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:20.211186 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f9452-e325-4193-8edb-34d13205379d None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:20.211592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f9452-e325-4193-8edb-34d13205379d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:20.211822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd90345-7e16-4c30-910c-dbd93e3aa1bf None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:20.212065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f9452-e325-4193-8edb-34d13205379d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:20.212312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd90345-7e16-4c30-910c-dbd93e3aa1bf None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:20.212795 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd90345-7e16-4c30-910c-dbd93e3aa1bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:20.258719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd90345-7e16-4c30-910c-dbd93e3aa1bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:20.261706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd90345-7e16-4c30-910c-dbd93e3aa1bf None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:20.279619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-671f9452-e325-4193-8edb-34d13205379d None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:20.281739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-671f9452-e325-4193-8edb-34d13205379d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:20.318725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-0cd90345-7e16-4c30-910c-dbd93e3aa1bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:20.318725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:20.318725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:20.318725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:20.318725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:20.318725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:20.318725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:20.318725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:20.318725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:20.318725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:20.318725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:20.318725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:20.318725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:20.318725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:20.318725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:20.318725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:20.321385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:20.321385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:20.321385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:20.321385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:20.321385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:20.321385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:20.321385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:20.321385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:20.321385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:20.321385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:20.321385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:20.321385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:20.321385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:20.321385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:20.321385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:20.321385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:20.321385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:20.323919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:20.323919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:20.323919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:20.323919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 270/547] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 482 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:20.330371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd90345-7e16-4c30-910c-dbd93e3aa1bf None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:20.484818 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 278/548] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:20.571507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:20.600166 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd90345-7e16-4c30-910c-dbd93e3aa1bf None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:20.661641 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fdb2c-1354-4ce8-aacc-80fcd34e5be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:20.662669 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fdb2c-1354-4ce8-aacc-80fcd34e5be5 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:20.663201 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fdb2c-1354-4ce8-aacc-80fcd34e5be5 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:20.663642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fdb2c-1354-4ce8-aacc-80fcd34e5be5 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:20.664765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72fdb2c-1354-4ce8-aacc-80fcd34e5be5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:20.665288 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72fdb2c-1354-4ce8-aacc-80fcd34e5be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:20.667899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72fdb2c-1354-4ce8-aacc-80fcd34e5be5 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:20.691920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 271/549] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:20.700305 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f9452-e325-4193-8edb-34d13205379d None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:20.923357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:20.984791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f9452-e325-4193-8edb-34d13205379d None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:21.079758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e99b0-16ed-4246-b310-89808c532605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:21.081686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e99b0-16ed-4246-b310-89808c532605 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:21.082459 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e99b0-16ed-4246-b310-89808c532605 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:21.083244 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e99b0-16ed-4246-b310-89808c532605 None admin] PATH_INFO: `/v3/users/swift` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:21.088000 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f0481-4ef9-4d69-9f9a-43c4a4c4cc18 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:21.088794 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f0481-4ef9-4d69-9f9a-43c4a4c4cc18 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:21.088974 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f0481-4ef9-4d69-9f9a-43c4a4c4cc18 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:21.091610 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 272/550] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:21.092913 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f0481-4ef9-4d69-9f9a-43c4a4c4cc18 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:21.102175 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e99b0-16ed-4246-b310-89808c532605 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:21.105463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e99b0-16ed-4246-b310-89808c532605 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:21.130080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-299e99b0-16ed-4246-b310-89808c532605 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:09:21.130080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:21.130080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:21.130080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:21.130080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:21.130080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:21.130080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:21.130080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:21.130080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:21.130080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:21.130080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:21.130080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:21.130080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:21.130080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:21.130080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:21.130080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:21.131934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:21.133759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:21.133759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:21.133759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:21.133759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:21.133759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:21.133759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:21.133759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:21.133759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:21.133759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:09:21.133759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:21.138096 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 279/551] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:09:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 439 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:21.140747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d069582-d2bf-4ddb-9655-de68920f4060 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:21.141625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d069582-d2bf-4ddb-9655-de68920f4060 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:21.142359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d069582-d2bf-4ddb-9655-de68920f4060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:21.176068 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-4d069582-d2bf-4ddb-9655-de68920f4060 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:21.178496 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-4d069582-d2bf-4ddb-9655-de68920f4060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:21.295411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f0481-4ef9-4d69-9f9a-43c4a4c4cc18 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:21.310779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 280/552] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:21.312340 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d069582-d2bf-4ddb-9655-de68920f4060 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:21.412132 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9781f48-fe7b-4a6c-af8f-55f6f2793f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:21.413118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9781f48-fe7b-4a6c-af8f-55f6f2793f91 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:21.413563 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9781f48-fe7b-4a6c-af8f-55f6f2793f91 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:21.414006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9781f48-fe7b-4a6c-af8f-55f6f2793f91 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:21.452540 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9781f48-fe7b-4a6c-af8f-55f6f2793f91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:21.462674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-f9781f48-fe7b-4a6c-af8f-55f6f2793f91 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:21.462674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:21.462674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:21.462674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:21.462674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:21.462674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:21.462674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:21.462674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:21.462674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:21.462674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:21.462674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:21.462674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:21.462674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:21.462674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:21.462674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:21.462674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:21.464572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:21.464572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:21.464572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:21.464572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:21.464572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:21.464572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:21.464572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:21.464572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:21.464572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:21.464572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:21.464572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:21.464572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:21.464572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:21.464572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:21.464572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:21.464572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:21.466232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:21.466232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:21.466232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:21.466232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:21.466232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:21.466232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:21.466232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:21.466232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:21.466232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:21.466232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:21.466232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:21.466232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:21.466232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 273/553] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:21.479133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9781f48-fe7b-4a6c-af8f-55f6f2793f91 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:21.498898 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:21.511021 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:21.623957 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d069582-d2bf-4ddb-9655-de68920f4060 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:21.682959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355d416-2f9c-494a-a45c-8219ee22134b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:21.684034 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355d416-2f9c-494a-a45c-8219ee22134b None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:21.684566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355d416-2f9c-494a-a45c-8219ee22134b None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:21.685199 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355d416-2f9c-494a-a45c-8219ee22134b None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:21.686158 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355d416-2f9c-494a-a45c-8219ee22134b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:21.686730 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355d416-2f9c-494a-a45c-8219ee22134b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:21.688681 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355d416-2f9c-494a-a45c-8219ee22134b None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:21.749880 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 281/554] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Sep 7 13:09:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:21.759415 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355d416-2f9c-494a-a45c-8219ee22134b None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:21.799714 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9781f48-fe7b-4a6c-af8f-55f6f2793f91 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:21.937963 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bb8a1-c9d6-4cb8-ae2c-823ca4e69939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:21.939458 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bb8a1-c9d6-4cb8-ae2c-823ca4e69939 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:21.939935 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bb8a1-c9d6-4cb8-ae2c-823ca4e69939 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:21.940391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bb8a1-c9d6-4cb8-ae2c-823ca4e69939 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:21.941683 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bb8a1-c9d6-4cb8-ae2c-823ca4e69939 None admin] RBAC: Adding query filter params (name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:21.942191 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bb8a1-c9d6-4cb8-ae2c-823ca4e69939 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:21.944722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bb8a1-c9d6-4cb8-ae2c-823ca4e69939 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:21.964219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 274/555] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:09:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 488 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:21.967297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02fb10-13d6-411d-8a30-7c1ce653fa8b None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:21.967880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02fb10-13d6-411d-8a30-7c1ce653fa8b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:21.968359 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02fb10-13d6-411d-8a30-7c1ce653fa8b None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:21.970722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 275/556] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:21.977626 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02fb10-13d6-411d-8a30-7c1ce653fa8b None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:21.990937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355d416-2f9c-494a-a45c-8219ee22134b None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:22.038150 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:22.054931 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66679cb5-c404-482e-a3e7-54965d414a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:22.056027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66679cb5-c404-482e-a3e7-54965d414a7f None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:22.056521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66679cb5-c404-482e-a3e7-54965d414a7f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:22.058871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66679cb5-c404-482e-a3e7-54965d414a7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:22.076568 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:22.120866 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02fb10-13d6-411d-8a30-7c1ce653fa8b None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:22.176736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be25616-8613-438b-8aeb-294f5989f119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:22.177385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be25616-8613-438b-8aeb-294f5989f119 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:22.177669 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be25616-8613-438b-8aeb-294f5989f119 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:22.177935 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be25616-8613-438b-8aeb-294f5989f119 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:22.179613 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66679cb5-c404-482e-a3e7-54965d414a7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:22.181129 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66679cb5-c404-482e-a3e7-54965d414a7f None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:22.192253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be25616-8613-438b-8aeb-294f5989f119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:22.193740 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be25616-8613-438b-8aeb-294f5989f119 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:22.214225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-9be25616-8613-438b-8aeb-294f5989f119 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:22.214225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:22.214225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:22.214225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:22.214225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:22.214225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:22.214225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:22.214225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:22.214225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:22.214225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:22.214225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:22.214225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:22.214225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:22.214225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:22.214225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:22.214225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:22.221253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:22.221253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:22.221253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:22.221253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:22.221253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:22.221253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:22.221253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:22.221253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:22.221253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:22.221253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:22.221253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:22.221253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:22.221253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:22.221253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:22.221253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:22.221253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:22.221253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:22.222330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:22.222330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:22.222330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:22.222330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 276/557] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:22.222330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea2378-b219-4e04-b9da-ce0d20671e49 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:22.222330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea2378-b219-4e04-b9da-ce0d20671e49 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:22.222330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea2378-b219-4e04-b9da-ce0d20671e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:22.237828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-4dea2378-b219-4e04-b9da-ce0d20671e49 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:22.239239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-4dea2378-b219-4e04-b9da-ce0d20671e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:22.367083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 282/558] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:09:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:09:22.367083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66679cb5-c404-482e-a3e7-54965d414a7f None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:22.379571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 277/559] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:22.381085 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea2378-b219-4e04-b9da-ce0d20671e49 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:22.418931 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:22.493158 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:22.567543 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66679cb5-c404-482e-a3e7-54965d414a7f None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:22.612134 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea2378-b219-4e04-b9da-ce0d20671e49 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:22.640786 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3f18f-ebaa-4042-8a0d-c6bbf59191c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:22.643183 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3f18f-ebaa-4042-8a0d-c6bbf59191c0 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:22.643740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3f18f-ebaa-4042-8a0d-c6bbf59191c0 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:22.644911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3f18f-ebaa-4042-8a0d-c6bbf59191c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:22.661386 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3f18f-ebaa-4042-8a0d-c6bbf59191c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:22.669041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-60b3f18f-ebaa-4042-8a0d-c6bbf59191c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:22.669041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:22.669041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:22.669041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:22.669041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:22.669041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:22.669041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:22.669041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:22.669041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:22.669041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:22.669041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:22.669041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:22.669041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:22.669041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:22.669041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:22.669041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:22.670078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:22.670078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:22.670078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:22.670078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:22.670078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:22.670078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:22.670078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:22.670078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:22.670078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:22.670078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:22.670078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:22.670078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:22.670078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:22.670078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:22.670078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:22.670078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:22.671101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:22.671101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:22.671101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:22.671101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:22.671101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:22.671101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:22.671101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:22.671101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:22.671101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:22.671101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:22.671101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:22.671101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:22.675510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 283/560] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:22.677008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3f18f-ebaa-4042-8a0d-c6bbf59191c0 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:22.685986 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6460962-a81f-4009-a5b1-a79c013e8409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:22.687077 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6460962-a81f-4009-a5b1-a79c013e8409 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:22.687706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6460962-a81f-4009-a5b1-a79c013e8409 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:22.688410 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6460962-a81f-4009-a5b1-a79c013e8409 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:22.689494 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6460962-a81f-4009-a5b1-a79c013e8409 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:22.690077 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6460962-a81f-4009-a5b1-a79c013e8409 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:22.692538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6460962-a81f-4009-a5b1-a79c013e8409 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:22.716921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 278/561] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:22.717249 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6460962-a81f-4009-a5b1-a79c013e8409 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:22.846603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:22.871582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3f18f-ebaa-4042-8a0d-c6bbf59191c0 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:22.899060 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6460962-a81f-4009-a5b1-a79c013e8409 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:22.912948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:22.931757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed80861-b442-4f93-b22b-51f2f1b08700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:22.932408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed80861-b442-4f93-b22b-51f2f1b08700 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:22.932695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed80861-b442-4f93-b22b-51f2f1b08700 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:22.932969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed80861-b442-4f93-b22b-51f2f1b08700 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:22.945356 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed80861-b442-4f93-b22b-51f2f1b08700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:22.946840 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed80861-b442-4f93-b22b-51f2f1b08700 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:22.948190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083e15b-743d-4a18-94fe-683c28fc88ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:22.949255 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083e15b-743d-4a18-94fe-683c28fc88ea None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:22.949750 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083e15b-743d-4a18-94fe-683c28fc88ea None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:22.950239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083e15b-743d-4a18-94fe-683c28fc88ea None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:22.951495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083e15b-743d-4a18-94fe-683c28fc88ea None admin] RBAC: Adding query filter params (name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:22.952007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083e15b-743d-4a18-94fe-683c28fc88ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:22.954581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083e15b-743d-4a18-94fe-683c28fc88ea None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:22.964016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-4ed80861-b442-4f93-b22b-51f2f1b08700 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:22.964016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:22.964016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:22.964016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:22.964016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:22.964016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:22.964016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:22.964016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:22.964016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:22.964016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:22.964016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:22.964016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:22.964016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:22.964016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:22.964016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:22.964016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:22.971611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:22.971611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:22.971611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:22.971611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:22.971611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:22.971611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:22.971611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:22.971611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:22.971611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:22.971611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:22.971611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:22.971611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:22.971611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:22.971611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:22.971611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:22.971611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:22.971611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:22.972612 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:22.972612 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:22.972612 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:22.972612 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 284/562] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:22.972612 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed80861-b442-4f93-b22b-51f2f1b08700 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:22.972905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 279/563] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:09:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:22.974140 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c42a0-8155-448d-9824-469487a487cd None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:22.974806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c42a0-8155-448d-9824-469487a487cd None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:22.975333 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c42a0-8155-448d-9824-469487a487cd None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:22.979053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 280/564] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:22.979584 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c42a0-8155-448d-9824-469487a487cd None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:23.121918 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c42a0-8155-448d-9824-469487a487cd None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:23.127435 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed80861-b442-4f93-b22b-51f2f1b08700 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:23.171123 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12676c13-84d3-40a3-aa01-d218d5030a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:23.171391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea7ab1-ce49-4a70-a09b-e681eeeb3120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:23.171832 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676c13-84d3-40a3-aa01-d218d5030a4a None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:23.172181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676c13-84d3-40a3-aa01-d218d5030a4a None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:23.172361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea7ab1-ce49-4a70-a09b-e681eeeb3120 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:23.173122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676c13-84d3-40a3-aa01-d218d5030a4a None admin] PATH_INFO: `/v3/users/glance` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:23.173339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea7ab1-ce49-4a70-a09b-e681eeeb3120 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:23.173499 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea7ab1-ce49-4a70-a09b-e681eeeb3120 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:23.177164 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:23.183360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12676c13-84d3-40a3-aa01-d218d5030a4a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:23.184888 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12676c13-84d3-40a3-aa01-d218d5030a4a None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:23.190214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea7ab1-ce49-4a70-a09b-e681eeeb3120 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:23.196190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-09ea7ab1-ce49-4a70-a09b-e681eeeb3120 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:23.196190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:23.196190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:23.196190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:23.196190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:23.196190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:23.196190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:23.196190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:23.196190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:23.196190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:23.196190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:23.196190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:23.196190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:23.196190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:23.196190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:23.196190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:23.197545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:23.197545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:23.197545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:23.197545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:23.197545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:23.197545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:23.197545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:23.197545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:23.197545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:23.197545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:23.197545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:23.197545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:23.197545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:23.197545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:23.197545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:23.197545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:23.199251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:23.199251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:23.199251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:23.199251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:23.199251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:23.199251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:23.199251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:23.199251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:23.199251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:23.199251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:23.199251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:23.199251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:23.199251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 281/565] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:23.199251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea7ab1-ce49-4a70-a09b-e681eeeb3120 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:23.207708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-12676c13-84d3-40a3-aa01-d218d5030a4a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 13:09:23.207708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:23.207708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:23.207708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:23.207708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:23.207708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:23.207708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:23.207708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:23.207708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:23.207708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:23.207708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:23.207708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:23.207708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:23.207708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:23.207708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:23.207708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:23.209648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:23.211788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:23.211788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:23.211788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:23.211788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:23.211788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:23.211788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:23.211788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:23.211788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:23.211788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 13:09:23.211788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:23.211788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 285/566] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:09:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:23.211788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12676c13-84d3-40a3-aa01-d218d5030a4a None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:23.248052 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:23.348914 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea7ab1-ce49-4a70-a09b-e681eeeb3120 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:23.368739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12676c13-84d3-40a3-aa01-d218d5030a4a None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:23.397962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e150c3e-2e16-4b6c-ae50-eb5cb012645d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:23.398576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e150c3e-2e16-4b6c-ae50-eb5cb012645d None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:23.398859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e150c3e-2e16-4b6c-ae50-eb5cb012645d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:23.399223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e150c3e-2e16-4b6c-ae50-eb5cb012645d None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:23.399828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e150c3e-2e16-4b6c-ae50-eb5cb012645d None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:23.400134 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e150c3e-2e16-4b6c-ae50-eb5cb012645d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:23.401586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e150c3e-2e16-4b6c-ae50-eb5cb012645d None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:23.415410 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 282/567] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:23.416764 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32968de-1e9a-409f-8708-9f160a072d57 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:23.417194 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32968de-1e9a-409f-8708-9f160a072d57 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:23.417445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32968de-1e9a-409f-8708-9f160a072d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:23.425867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b24ec-d232-44d6-aa67-da2b09306021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:23.426440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b24ec-d232-44d6-aa67-da2b09306021 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:23.426715 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b24ec-d232-44d6-aa67-da2b09306021 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:23.427002 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b24ec-d232-44d6-aa67-da2b09306021 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:23.439988 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b24ec-d232-44d6-aa67-da2b09306021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:23.441809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b24ec-d232-44d6-aa67-da2b09306021 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:23.445411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c32968de-1e9a-409f-8708-9f160a072d57 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:23.446656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c32968de-1e9a-409f-8708-9f160a072d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:23.460890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-5c3b24ec-d232-44d6-aa67-da2b09306021 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:23.460890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:23.460890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:23.460890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:23.460890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:23.460890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:23.460890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:23.460890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:23.460890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:23.460890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:23.460890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:23.460890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:23.460890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:23.460890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:23.460890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:23.460890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:23.462871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:23.462871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:23.462871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:23.462871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:23.462871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:23.462871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:23.462871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:23.462871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:23.462871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:23.462871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:23.462871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:23.462871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:23.462871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:23.462871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:23.462871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:23.462871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:23.462871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:23.464584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:23.464584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:23.464584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:23.464584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 286/568] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:23.464584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b24ec-d232-44d6-aa67-da2b09306021 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:23.532556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:23.559716 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 283/569] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:23.559977 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32968de-1e9a-409f-8708-9f160a072d57 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:23.614742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b24ec-d232-44d6-aa67-da2b09306021 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:23.615440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:23.691107 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e0177-1c2e-42ed-a97a-4aa930ab8dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:23.691848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e0177-1c2e-42ed-a97a-4aa930ab8dee None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:23.692177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e0177-1c2e-42ed-a97a-4aa930ab8dee None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:23.692546 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e0177-1c2e-42ed-a97a-4aa930ab8dee None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:23.693347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e0177-1c2e-42ed-a97a-4aa930ab8dee None admin] RBAC: Adding query filter params (name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:23.693655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e0177-1c2e-42ed-a97a-4aa930ab8dee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:23.695399 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e0177-1c2e-42ed-a97a-4aa930ab8dee None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:23.713891 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 287/570] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:09:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:23.713891 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e0177-1c2e-42ed-a97a-4aa930ab8dee None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:23.760550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32968de-1e9a-409f-8708-9f160a072d57 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:23.833542 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9826f8-c1ac-455b-a8c3-e782f51ade82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:23.834598 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9826f8-c1ac-455b-a8c3-e782f51ade82 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:23.835172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9826f8-c1ac-455b-a8c3-e782f51ade82 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:23.835680 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9826f8-c1ac-455b-a8c3-e782f51ade82 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:23.836965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9826f8-c1ac-455b-a8c3-e782f51ade82 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:23.837430 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9826f8-c1ac-455b-a8c3-e782f51ade82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:23.839949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9826f8-c1ac-455b-a8c3-e782f51ade82 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:23.867496 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 284/571] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Sep 7 13:09:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:23.869595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9826f8-c1ac-455b-a8c3-e782f51ade82 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:23.915133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e0177-1c2e-42ed-a97a-4aa930ab8dee None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:23.931942 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:23.964181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c74e83-3f41-4918-bcc7-2f80fb3df2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:23.964837 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c74e83-3f41-4918-bcc7-2f80fb3df2a6 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:23.965130 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c74e83-3f41-4918-bcc7-2f80fb3df2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:23.965445 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c74e83-3f41-4918-bcc7-2f80fb3df2a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:23.976793 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c74e83-3f41-4918-bcc7-2f80fb3df2a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:23.979624 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c74e83-3f41-4918-bcc7-2f80fb3df2a6 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:24.003804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-d7c74e83-3f41-4918-bcc7-2f80fb3df2a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:24.003804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:24.003804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:24.003804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:24.003804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:24.003804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:24.003804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:24.003804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:24.003804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:24.003804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:24.003804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:24.003804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:24.003804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:24.003804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:24.003804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:24.005609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:24.005609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:24.005609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:24.005609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:24.005609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:24.005609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:24.005609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:24.005609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:24.005609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:24.005609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:24.005609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:24.005609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:24.005609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:24.005609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:24.005609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:24.005609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:24.005609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:24.007349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:24.007349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:24.007349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:24.007349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:24.007349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:24.007349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:24.007349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:24.007349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:24.007349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 288/572] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:24.008380 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c74e83-3f41-4918-bcc7-2f80fb3df2a6 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:24.017893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:24.040416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9826f8-c1ac-455b-a8c3-e782f51ade82 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:24.085733 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c802a-fad8-45e7-b8d7-eecd10da71ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:24.086616 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c802a-fad8-45e7-b8d7-eecd10da71ca None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:24.087089 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c802a-fad8-45e7-b8d7-eecd10da71ca None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:24.087529 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c802a-fad8-45e7-b8d7-eecd10da71ca None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:24.088625 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c802a-fad8-45e7-b8d7-eecd10da71ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:24.089111 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c802a-fad8-45e7-b8d7-eecd10da71ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:24.091350 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c802a-fad8-45e7-b8d7-eecd10da71ca None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:24.110886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 285/573] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:24.112283 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c802a-fad8-45e7-b8d7-eecd10da71ca None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:24.181695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c74e83-3f41-4918-bcc7-2f80fb3df2a6 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:24.249118 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b468b0e-1a56-44d6-bd2a-970956ab5f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:24.250050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b468b0e-1a56-44d6-bd2a-970956ab5f34 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:24.250721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b468b0e-1a56-44d6-bd2a-970956ab5f34 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:24.251943 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b468b0e-1a56-44d6-bd2a-970956ab5f34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:24.259927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c802a-fad8-45e7-b8d7-eecd10da71ca None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:24.272308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b468b0e-1a56-44d6-bd2a-970956ab5f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:24.275029 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b468b0e-1a56-44d6-bd2a-970956ab5f34 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:24.303772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-1b468b0e-1a56-44d6-bd2a-970956ab5f34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:24.303772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:24.303772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:24.303772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:24.303772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:24.303772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:24.303772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:24.303772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:24.303772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:24.303772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:24.303772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:24.303772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:24.303772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:24.303772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:24.303772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:24.303772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:24.305809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:24.305809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:24.305809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:24.305809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:24.305809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:24.305809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:24.305809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:24.305809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:24.305809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:24.305809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:24.305809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:24.305809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:24.305809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:24.305809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:24.305809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:24.305809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:24.305809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:24.306939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:24.306939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:24.306939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:24.306939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 289/574] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:24.306939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b468b0e-1a56-44d6-bd2a-970956ab5f34 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:24.321493 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e555e-cf80-4b6a-8eb6-1403fc341411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:24.322544 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e555e-cf80-4b6a-8eb6-1403fc341411 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:24.323013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e555e-cf80-4b6a-8eb6-1403fc341411 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:24.323531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e555e-cf80-4b6a-8eb6-1403fc341411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:24.341731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:24.345826 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e555e-cf80-4b6a-8eb6-1403fc341411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:24.348747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e555e-cf80-4b6a-8eb6-1403fc341411 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:24.354186 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:24.377882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-962e555e-cf80-4b6a-8eb6-1403fc341411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:24.377882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:24.377882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:24.377882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:24.377882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:24.377882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:24.377882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:24.377882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:24.377882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:24.377882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:24.377882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:24.377882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:24.377882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:24.377882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:24.377882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:24.377882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:24.379877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:24.379877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:24.379877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:24.379877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:24.379877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:24.379877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:24.379877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:24.379877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:24.379877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:24.379877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:24.379877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:24.379877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:24.379877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:24.379877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:24.379877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:24.379877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:24.379877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:24.382362 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:24.382362 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:24.382362 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:24.382362 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 286/575] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:24.382362 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e555e-cf80-4b6a-8eb6-1403fc341411 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:24.475425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b468b0e-1a56-44d6-bd2a-970956ab5f34 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:24.543193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3581a4-2df6-4e32-94c8-7142eeab32e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:24.544305 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3581a4-2df6-4e32-94c8-7142eeab32e2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:24.544818 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3581a4-2df6-4e32-94c8-7142eeab32e2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:24.545374 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3581a4-2df6-4e32-94c8-7142eeab32e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:24.565993 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3581a4-2df6-4e32-94c8-7142eeab32e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:24.566454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e555e-cf80-4b6a-8eb6-1403fc341411 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:24.575556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-ec3581a4-2df6-4e32-94c8-7142eeab32e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:24.575556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:24.575556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:24.575556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:24.575556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:24.575556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:24.575556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:24.575556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:24.575556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:24.575556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:24.575556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:24.575556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:24.575556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:24.575556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:24.575556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:24.575556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:24.577752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:24.577752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:24.577752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:24.577752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:24.577752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:24.577752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:24.577752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:24.577752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:24.577752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:24.577752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:24.577752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:24.577752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:24.577752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:24.577752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:24.577752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:24.577752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:24.579476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:24.579476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:24.579476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:24.579476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:24.579476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:24.579476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:24.579476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:24.579476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:24.579476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:24.579476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:24.579476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:24.579476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:24.579476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 290/576] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:24.580869 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3581a4-2df6-4e32-94c8-7142eeab32e2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:24.630141 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d55c70-6128-43f1-a052-0cf2b50cf232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:24.631045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d55c70-6128-43f1-a052-0cf2b50cf232 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:24.631359 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d55c70-6128-43f1-a052-0cf2b50cf232 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:24.631777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d55c70-6128-43f1-a052-0cf2b50cf232 None admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:24.632692 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d55c70-6128-43f1-a052-0cf2b50cf232 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:24.633148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d55c70-6128-43f1-a052-0cf2b50cf232 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:24.634837 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d55c70-6128-43f1-a052-0cf2b50cf232 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:24.651911 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 287/577] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:24.653410 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d55c70-6128-43f1-a052-0cf2b50cf232 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:24.727512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:24.731960 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:24.760816 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3581a4-2df6-4e32-94c8-7142eeab32e2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:24.804159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d55c70-6128-43f1-a052-0cf2b50cf232 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:24.825588 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cb659-7575-42b8-bc4b-bd6c677d44b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:24.826430 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cb659-7575-42b8-bc4b-bd6c677d44b9 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:24.826748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cb659-7575-42b8-bc4b-bd6c677d44b9 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:24.827122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cb659-7575-42b8-bc4b-bd6c677d44b9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:24.837469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cb659-7575-42b8-bc4b-bd6c677d44b9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:24.839158 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cb659-7575-42b8-bc4b-bd6c677d44b9 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:24.854959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-238cb659-7575-42b8-bc4b-bd6c677d44b9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 13:09:24.854959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:24.854959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:24.854959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:24.854959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:24.854959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:24.854959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:24.854959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:24.854959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:24.854959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:24.854959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:24.854959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:24.854959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:24.854959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:24.854959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:24.854959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:24.865475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:24.867364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:24.867364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:24.867364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:24.867364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:24.867364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:24.867364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:24.867364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:24.867364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:24.867364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 13:09:24.867364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:24.867364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 291/578] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:09:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:24.867364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cb659-7575-42b8-bc4b-bd6c677d44b9 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:24.868683 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ac4d9-5653-471e-ac44-df3a0d157a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:24.868683 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ac4d9-5653-471e-ac44-df3a0d157a81 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:24.868683 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ac4d9-5653-471e-ac44-df3a0d157a81 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:24.868683 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ac4d9-5653-471e-ac44-df3a0d157a81 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:24.869189 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ac4d9-5653-471e-ac44-df3a0d157a81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:24.869189 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ac4d9-5653-471e-ac44-df3a0d157a81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:24.869189 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ac4d9-5653-471e-ac44-df3a0d157a81 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:24.890821 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 288/579] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:24.892729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ac4d9-5653-471e-ac44-df3a0d157a81 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:25.059201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cb659-7575-42b8-bc4b-bd6c677d44b9 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:25.059770 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ac4d9-5653-471e-ac44-df3a0d157a81 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:25.077772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:25.113010 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b050ed-e108-462f-960d-cf1e22c8802b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:25.113741 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b050ed-e108-462f-960d-cf1e22c8802b None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:25.114090 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b050ed-e108-462f-960d-cf1e22c8802b None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:25.114481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b050ed-e108-462f-960d-cf1e22c8802b None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:25.115334 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b050ed-e108-462f-960d-cf1e22c8802b None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:25.115717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b050ed-e108-462f-960d-cf1e22c8802b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:25.117470 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b050ed-e108-462f-960d-cf1e22c8802b None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:25.117897 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d559ebb-ae44-497d-baa9-4d64cc1a0de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:25.118878 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d559ebb-ae44-497d-baa9-4d64cc1a0de7 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:25.119768 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d559ebb-ae44-497d-baa9-4d64cc1a0de7 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:25.120748 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d559ebb-ae44-497d-baa9-4d64cc1a0de7 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:25.121845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d559ebb-ae44-497d-baa9-4d64cc1a0de7 None admin] RBAC: Adding query filter params (name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:25.122388 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d559ebb-ae44-497d-baa9-4d64cc1a0de7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:25.125176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d559ebb-ae44-497d-baa9-4d64cc1a0de7 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:25.133111 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 292/580] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:25.133993 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b050ed-e108-462f-960d-cf1e22c8802b None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:25.137067 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:25.145482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 289/581] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:09:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:25.151827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d559ebb-ae44-497d-baa9-4d64cc1a0de7 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:25.306164 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b050ed-e108-462f-960d-cf1e22c8802b None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:25.310912 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d559ebb-ae44-497d-baa9-4d64cc1a0de7 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:25.353352 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceceab7-c223-4255-8d66-1e61a7ea4f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:25.354092 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceceab7-c223-4255-8d66-1e61a7ea4f25 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:25.354378 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceceab7-c223-4255-8d66-1e61a7ea4f25 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:25.354670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceceab7-c223-4255-8d66-1e61a7ea4f25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:25.355656 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceceab7-c223-4255-8d66-1e61a7ea4f25 None admin] RBAC: Adding query filter params (role.id=743ed2d8e7114975bd9245742a41fe0f, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=5cedc7091d2f46b5be68818d26399e9b) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:25.355960 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceceab7-c223-4255-8d66-1e61a7ea4f25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:25.357442 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceceab7-c223-4255-8d66-1e61a7ea4f25 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:25.364526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 293/582] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:09:25 2022] GET /identity/v3/role_assignments?user.id=5cedc7091d2f46b5be68818d26399e9b&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=743ed2d8e7114975bd9245742a41fe0f => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:25.365393 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceceab7-c223-4255-8d66-1e61a7ea4f25 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:25.372114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d6acb-a78d-4017-9f19-7307577029f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:25.373185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d6acb-a78d-4017-9f19-7307577029f5 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:25.373689 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d6acb-a78d-4017-9f19-7307577029f5 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:25.374166 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d6acb-a78d-4017-9f19-7307577029f5 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:25.376221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d6acb-a78d-4017-9f19-7307577029f5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:25.376767 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d6acb-a78d-4017-9f19-7307577029f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:25.379755 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d6acb-a78d-4017-9f19-7307577029f5 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:25.409123 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 290/583] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Sep 7 13:09:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:25.410895 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d6acb-a78d-4017-9f19-7307577029f5 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:25.487531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:25.513996 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:25.583758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceceab7-c223-4255-8d66-1e61a7ea4f25 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:25.635697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d6acb-a78d-4017-9f19-7307577029f5 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:25.641612 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d2255-5505-45d8-9dab-dcb07dbc76e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:25.642308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d2255-5505-45d8-9dab-dcb07dbc76e2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:25.642583 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d2255-5505-45d8-9dab-dcb07dbc76e2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:25.642853 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d2255-5505-45d8-9dab-dcb07dbc76e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:25.655818 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d2255-5505-45d8-9dab-dcb07dbc76e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:25.658106 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d2255-5505-45d8-9dab-dcb07dbc76e2 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:25.677918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-8a7d2255-5505-45d8-9dab-dcb07dbc76e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:09:25.677918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:25.677918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:25.677918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:25.677918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:25.677918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:25.677918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:25.677918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:25.677918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:25.677918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:25.677918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:25.677918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:25.677918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:25.677918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:25.677918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:25.677918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:25.686920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:25.689100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:25.689100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:25.689100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:25.689100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:25.689100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:25.689100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:25.689100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:25.689100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:25.689100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:09:25.689100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:25.689100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 294/584] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 7 13:09:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:25.689100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d2255-5505-45d8-9dab-dcb07dbc76e2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:25.717390 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703c4c4-fbfc-49db-a6bd-7678afa05952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:25.718561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703c4c4-fbfc-49db-a6bd-7678afa05952 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:25.719334 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703c4c4-fbfc-49db-a6bd-7678afa05952 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:25.721286 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703c4c4-fbfc-49db-a6bd-7678afa05952 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:25.734339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5703c4c4-fbfc-49db-a6bd-7678afa05952 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:25.738118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5703c4c4-fbfc-49db-a6bd-7678afa05952 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:25.755805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-5703c4c4-fbfc-49db-a6bd-7678afa05952 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:25.755805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:25.755805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:25.755805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:25.755805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:25.755805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:25.755805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:25.755805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:25.755805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:25.755805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:25.755805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:25.755805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:25.755805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:25.755805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:25.755805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:25.757116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:25.757116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:25.757116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:25.757116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:25.757116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:25.757116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:25.757116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:25.757116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:25.757116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:25.757116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:25.757116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:25.757116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:25.757116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:25.757116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:25.757116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:25.757116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:25.757116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:25.758226 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:25.758226 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:25.758226 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:25.758226 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:25.758226 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:25.758226 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:25.758226 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:25.758226 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:25.763578 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 291/585] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:25.765297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703c4c4-fbfc-49db-a6bd-7678afa05952 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:25.855778 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d2255-5505-45d8-9dab-dcb07dbc76e2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:25.900526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:25.912595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15997b8b-5efb-4b07-b56d-6fb35afbe5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:25.913868 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15997b8b-5efb-4b07-b56d-6fb35afbe5b6 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:25.913868 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15997b8b-5efb-4b07-b56d-6fb35afbe5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:25.914264 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15997b8b-5efb-4b07-b56d-6fb35afbe5b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:25.928170 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15997b8b-5efb-4b07-b56d-6fb35afbe5b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:25.929959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15997b8b-5efb-4b07-b56d-6fb35afbe5b6 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:25.945238 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:25.961163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-15997b8b-5efb-4b07-b56d-6fb35afbe5b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:25.961163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:25.961163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:25.961163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:25.961163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:25.961163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:25.961163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:25.961163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:25.961163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:25.961163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:25.961163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:25.961163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:25.961163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:25.961163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:25.961163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:25.961163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:25.962526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:25.962526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:25.962526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:25.962526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:25.962526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:25.962526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:25.962526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:25.962526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:25.962526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:25.962526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:25.962526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:25.962526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:25.962526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:25.962526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:25.962526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:25.962526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:25.962526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:25.963646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:25.963646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:25.963646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:25.963863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703c4c4-fbfc-49db-a6bd-7678afa05952 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:25.971058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 295/586] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:25.971058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15997b8b-5efb-4b07-b56d-6fb35afbe5b6 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:26.015671 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda81eab-2cb6-4f28-ba73-53f5c36c8368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:26.017229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81eab-2cb6-4f28-ba73-53f5c36c8368 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:26.018080 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81eab-2cb6-4f28-ba73-53f5c36c8368 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:26.018386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81eab-2cb6-4f28-ba73-53f5c36c8368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:26.039474 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda81eab-2cb6-4f28-ba73-53f5c36c8368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:26.041047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda81eab-2cb6-4f28-ba73-53f5c36c8368 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:26.058459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-eda81eab-2cb6-4f28-ba73-53f5c36c8368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:26.058459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:26.058459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:26.058459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:26.058459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:26.058459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:26.058459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:26.058459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:26.058459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:26.058459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:26.058459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:26.058459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:26.058459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:26.058459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:26.058459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:26.058459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:26.060008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:26.060008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:26.060008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:26.060008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:26.060008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:26.060008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:26.060008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:26.060008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:26.060008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:26.060008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:26.060008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:26.060008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:26.060008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:26.060008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:26.060008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:26.060008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:26.060008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:26.061812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:26.061812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:26.061812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:26.061812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 292/587] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:26.061812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda81eab-2cb6-4f28-ba73-53f5c36c8368 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:26.135399 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15997b8b-5efb-4b07-b56d-6fb35afbe5b6 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:26.186706 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93c2eb-3569-4b82-a952-758152b086f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:26.187664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93c2eb-3569-4b82-a952-758152b086f2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:26.188881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93c2eb-3569-4b82-a952-758152b086f2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:26.189231 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93c2eb-3569-4b82-a952-758152b086f2 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:26.191108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93c2eb-3569-4b82-a952-758152b086f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:26.191745 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93c2eb-3569-4b82-a952-758152b086f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:26.195060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93c2eb-3569-4b82-a952-758152b086f2 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:26.245086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 296/588] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 7 13:09:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:26.245086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93c2eb-3569-4b82-a952-758152b086f2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:26.264159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda81eab-2cb6-4f28-ba73-53f5c36c8368 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:26.316570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:26.328836 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b3eb5-5074-4698-9f1f-ee337ac1dc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:26.329820 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b3eb5-5074-4698-9f1f-ee337ac1dc9c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:26.329820 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b3eb5-5074-4698-9f1f-ee337ac1dc9c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:26.336200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b3eb5-5074-4698-9f1f-ee337ac1dc9c None admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:26.336200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b3eb5-5074-4698-9f1f-ee337ac1dc9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:26.336200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b3eb5-5074-4698-9f1f-ee337ac1dc9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:26.336200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b3eb5-5074-4698-9f1f-ee337ac1dc9c None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:26.351875 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 293/589] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:26.353945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b3eb5-5074-4698-9f1f-ee337ac1dc9c None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:26.364869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:26.455442 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93c2eb-3569-4b82-a952-758152b086f2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:26.531273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbde392-7b1d-4698-a85a-ecbdad67bfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:26.532337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbde392-7b1d-4698-a85a-ecbdad67bfcc None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:26.532973 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbde392-7b1d-4698-a85a-ecbdad67bfcc None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:26.533436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbde392-7b1d-4698-a85a-ecbdad67bfcc None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:26.534585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbde392-7b1d-4698-a85a-ecbdad67bfcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:26.535122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbde392-7b1d-4698-a85a-ecbdad67bfcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:26.537314 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbde392-7b1d-4698-a85a-ecbdad67bfcc None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:26.540433 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b3eb5-5074-4698-9f1f-ee337ac1dc9c None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:26.560863 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 297/590] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:26.560863 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbde392-7b1d-4698-a85a-ecbdad67bfcc None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:26.614646 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebe689-1807-4c08-80d0-b5421ff8565f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:26.615306 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebe689-1807-4c08-80d0-b5421ff8565f None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:26.615710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebe689-1807-4c08-80d0-b5421ff8565f None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:26.615964 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebe689-1807-4c08-80d0-b5421ff8565f None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:26.616624 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebe689-1807-4c08-80d0-b5421ff8565f None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:26.617342 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebe689-1807-4c08-80d0-b5421ff8565f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:26.618737 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebe689-1807-4c08-80d0-b5421ff8565f None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:26.644303 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 294/591] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:26.645617 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebe689-1807-4c08-80d0-b5421ff8565f None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:26.716487 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:26.745766 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbde392-7b1d-4698-a85a-ecbdad67bfcc None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:26.795874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:26.810631 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c0c07-10d2-4beb-9c5a-56b3f446ab97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:26.811590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c0c07-10d2-4beb-9c5a-56b3f446ab97 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:26.812015 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c0c07-10d2-4beb-9c5a-56b3f446ab97 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:26.812462 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c0c07-10d2-4beb-9c5a-56b3f446ab97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:26.831616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3c0c07-10d2-4beb-9c5a-56b3f446ab97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:26.833914 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3c0c07-10d2-4beb-9c5a-56b3f446ab97 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:26.861724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-eb3c0c07-10d2-4beb-9c5a-56b3f446ab97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:26.861724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:26.861724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:26.861724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:26.861724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:26.861724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:26.861724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:26.861724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:26.861724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:26.861724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:26.861724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:26.861724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:26.861724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:26.861724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:26.861724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:26.861724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:26.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:26.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:26.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:26.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:26.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:26.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:26.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:26.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:26.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:26.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:26.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:26.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:26.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:26.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:26.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:26.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:26.862751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:26.863723 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:26.863723 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:26.863723 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:26.870220 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 298/592] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:26.871723 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c0c07-10d2-4beb-9c5a-56b3f446ab97 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:26.880055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebe689-1807-4c08-80d0-b5421ff8565f None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:26.945074 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34871a3-5ca3-41be-a9b4-c60b81e831b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:26.945665 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34871a3-5ca3-41be-a9b4-c60b81e831b0 None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:26.945969 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34871a3-5ca3-41be-a9b4-c60b81e831b0 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:26.946241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34871a3-5ca3-41be-a9b4-c60b81e831b0 None admin] PATH_INFO: `/v3/projects/4574cf3ff270485faf60594019f7e05c/users/d1a48805d95640489d7ce3333dd687b9/roles/743ed2d8e7114975bd9245742a41fe0f` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:26.975058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34871a3-5ca3-41be-a9b4-c60b81e831b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=4574cf3ff270485faf60594019f7e05c, user_id=d1a48805d95640489d7ce3333dd687b9, role_id=743ed2d8e7114975bd9245742a41fe0f)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:26.977584 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34871a3-5ca3-41be-a9b4-c60b81e831b0 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:27.025811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c0c07-10d2-4beb-9c5a-56b3f446ab97 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:27.076751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8225af8-01e2-4d7c-be9c-b8ce4a830f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:27.077410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8225af8-01e2-4d7c-be9c-b8ce4a830f9f None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:27.077835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8225af8-01e2-4d7c-be9c-b8ce4a830f9f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:27.078003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8225af8-01e2-4d7c-be9c-b8ce4a830f9f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:27.084502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 295/593] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:09:26 2022] PUT /identity/v3/projects/4574cf3ff270485faf60594019f7e05c/users/d1a48805d95640489d7ce3333dd687b9/roles/743ed2d8e7114975bd9245742a41fe0f => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:09:27.085984 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34871a3-5ca3-41be-a9b4-c60b81e831b0 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:27.101762 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8225af8-01e2-4d7c-be9c-b8ce4a830f9f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:27.106624 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8225af8-01e2-4d7c-be9c-b8ce4a830f9f None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:27.131937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-c8225af8-01e2-4d7c-be9c-b8ce4a830f9f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 13:09:27.131937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:27.131937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:27.131937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:27.131937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:27.131937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:27.131937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:27.131937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:27.131937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:27.131937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:27.131937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:27.131937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:27.131937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:27.131937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:27.131937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:27.131937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:27.133422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:27.134570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:27.134570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:27.134570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:27.134570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:27.134570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:27.134570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:27.134570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:27.134570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:27.134570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 13:09:27.134570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:27.143187 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 299/594] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:27.143187 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8225af8-01e2-4d7c-be9c-b8ce4a830f9f None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:27.164740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:27.308262 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34871a3-5ca3-41be-a9b4-c60b81e831b0 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:27.327802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8225af8-01e2-4d7c-be9c-b8ce4a830f9f None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:27.335869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:27.378526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beda681-10e7-473a-8a0b-ed4b26480d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:27.379413 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beda681-10e7-473a-8a0b-ed4b26480d02 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:27.380133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beda681-10e7-473a-8a0b-ed4b26480d02 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:27.380597 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beda681-10e7-473a-8a0b-ed4b26480d02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:27.395357 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beda681-10e7-473a-8a0b-ed4b26480d02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:27.398940 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beda681-10e7-473a-8a0b-ed4b26480d02 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:27.401807 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef57a7-3aaa-458f-90ec-1fde2b35a656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:27.403521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef57a7-3aaa-458f-90ec-1fde2b35a656 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:27.404218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef57a7-3aaa-458f-90ec-1fde2b35a656 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:27.405458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef57a7-3aaa-458f-90ec-1fde2b35a656 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:27.407157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef57a7-3aaa-458f-90ec-1fde2b35a656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:27.407977 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef57a7-3aaa-458f-90ec-1fde2b35a656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:27.410912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef57a7-3aaa-458f-90ec-1fde2b35a656 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:27.423007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-2beda681-10e7-473a-8a0b-ed4b26480d02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:27.423007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:27.423007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:27.423007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:27.423007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:27.423007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:27.423007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:27.423007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:27.423007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:27.423007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:27.423007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:27.423007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:27.423007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:27.423007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:27.423007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:27.424119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:27.424119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:27.424119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:27.424119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:27.424119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:27.424119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:27.424119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:27.424119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:27.424119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:27.424119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:27.424119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:27.424119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:27.424119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:27.424119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:27.424119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:27.424119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:27.424119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:27.425170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:27.425170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:27.425170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:27.425170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:27.425170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:27.425170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:27.425170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:27.425170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:27.431201 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 296/595] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:27.433122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beda681-10e7-473a-8a0b-ed4b26480d02 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:27.439938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 300/596] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:27.444199 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef57a7-3aaa-458f-90ec-1fde2b35a656 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:27.621475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:27.697140 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beda681-10e7-473a-8a0b-ed4b26480d02 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:27.699174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef57a7-3aaa-458f-90ec-1fde2b35a656 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:27.775437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f693f-7604-4f25-9c3e-bf76b573667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:27.776653 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f693f-7604-4f25-9c3e-bf76b573667a None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:27.777143 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f693f-7604-4f25-9c3e-bf76b573667a None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:27.777476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e957dd4-12be-4bec-b4e0-65485863872a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:27.777683 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f693f-7604-4f25-9c3e-bf76b573667a None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:27.778866 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e957dd4-12be-4bec-b4e0-65485863872a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:27.779099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f693f-7604-4f25-9c3e-bf76b573667a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:27.779663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f693f-7604-4f25-9c3e-bf76b573667a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:27.780001 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e957dd4-12be-4bec-b4e0-65485863872a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:27.780861 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e957dd4-12be-4bec-b4e0-65485863872a None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:27.782905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e957dd4-12be-4bec-b4e0-65485863872a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:27.783303 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f693f-7604-4f25-9c3e-bf76b573667a None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:27.783874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e957dd4-12be-4bec-b4e0-65485863872a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:27.786767 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e957dd4-12be-4bec-b4e0-65485863872a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:27.822893 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 301/597] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:09:27.822893 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f693f-7604-4f25-9c3e-bf76b573667a None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:27.831737 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 297/598] 10.222.0.62 () {56 vars in 1139 bytes} [Wed Sep 7 13:09:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:27.833744 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e957dd4-12be-4bec-b4e0-65485863872a None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:27.917419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:28.050327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f693f-7604-4f25-9c3e-bf76b573667a None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:28.078461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e957dd4-12be-4bec-b4e0-65485863872a None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:28.113028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:28.131671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee39e1-7d07-45fe-a2dd-0e0e503c21fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:28.134066 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee39e1-7d07-45fe-a2dd-0e0e503c21fa None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:28.134066 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee39e1-7d07-45fe-a2dd-0e0e503c21fa None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:28.134553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee39e1-7d07-45fe-a2dd-0e0e503c21fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:28.154745 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee39e1-7d07-45fe-a2dd-0e0e503c21fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:28.157911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee39e1-7d07-45fe-a2dd-0e0e503c21fa None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:28.158322 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24156a9d-4f31-4418-8418-928f2ccc92ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:28.159803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24156a9d-4f31-4418-8418-928f2ccc92ab None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:28.160539 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24156a9d-4f31-4418-8418-928f2ccc92ab None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:28.161371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24156a9d-4f31-4418-8418-928f2ccc92ab None admin] PATH_INFO: `/v3/projects/4574cf3ff270485faf60594019f7e05c/users/fc2fa0fa46b842ceb1cac437166b8aea/roles/743ed2d8e7114975bd9245742a41fe0f` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:28.186533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-f1ee39e1-7d07-45fe-a2dd-0e0e503c21fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:28.186533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:28.186533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:28.186533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:28.186533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:28.186533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:28.186533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:28.186533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:28.186533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:28.186533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:28.186533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:28.186533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:28.186533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:28.186533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:28.186533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:28.189726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:28.189726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:28.189726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:28.189726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:28.189726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:28.189726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:28.189726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:28.189726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:28.189726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:28.189726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:28.189726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:28.189726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:28.189726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:28.189726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:28.189726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:28.189726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:28.189726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:28.192447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:28.192447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:28.192447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:28.192447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:28.192447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:28.192447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:28.192447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:28.192447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:28.196041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 302/599] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:28.197874 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee39e1-7d07-45fe-a2dd-0e0e503c21fa None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:28.199563 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24156a9d-4f31-4418-8418-928f2ccc92ab None admin] RBAC: Authorizing `identity:create_grant(project_id=4574cf3ff270485faf60594019f7e05c, user_id=fc2fa0fa46b842ceb1cac437166b8aea, role_id=743ed2d8e7114975bd9245742a41fe0f)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:28.202778 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24156a9d-4f31-4418-8418-928f2ccc92ab None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:28.296630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 298/600] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:09:27 2022] PUT /identity/v3/projects/4574cf3ff270485faf60594019f7e05c/users/fc2fa0fa46b842ceb1cac437166b8aea/roles/743ed2d8e7114975bd9245742a41fe0f => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:09:28.298205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24156a9d-4f31-4418-8418-928f2ccc92ab None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:28.424093 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee39e1-7d07-45fe-a2dd-0e0e503c21fa None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:28.427199 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:28.452196 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24156a9d-4f31-4418-8418-928f2ccc92ab None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:28.500071 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed353226-7e2b-4181-b665-54efee5711da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:28.501960 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed353226-7e2b-4181-b665-54efee5711da None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:28.502454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed353226-7e2b-4181-b665-54efee5711da None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:28.502941 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed353226-7e2b-4181-b665-54efee5711da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:28.505534 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcfbde-78a9-4c7d-bab5-91bc7af5268a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:28.506688 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcfbde-78a9-4c7d-bab5-91bc7af5268a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:28.507307 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcfbde-78a9-4c7d-bab5-91bc7af5268a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:28.507694 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcfbde-78a9-4c7d-bab5-91bc7af5268a None admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:28.509340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfcfbde-78a9-4c7d-bab5-91bc7af5268a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:28.509789 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfcfbde-78a9-4c7d-bab5-91bc7af5268a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:28.512297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfcfbde-78a9-4c7d-bab5-91bc7af5268a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:28.531258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 299/601] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:28.533337 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101d3b-db6d-4bdc-b5f1-ba7766eba326 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:28.533774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101d3b-db6d-4bdc-b5f1-ba7766eba326 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:28.534053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101d3b-db6d-4bdc-b5f1-ba7766eba326 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:28.535896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed353226-7e2b-4181-b665-54efee5711da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:28.537919 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed353226-7e2b-4181-b665-54efee5711da None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:28.540065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 300/602] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:28.543076 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d101d3b-db6d-4bdc-b5f1-ba7766eba326 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:28.560721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-ed353226-7e2b-4181-b665-54efee5711da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:28.560721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:28.560721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:28.560721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:28.560721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:28.560721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:28.560721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:28.560721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:28.560721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:28.560721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:28.560721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:28.560721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:28.560721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:28.560721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:28.560721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:28.560721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:28.561761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:28.561761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:28.561761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:28.561761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:28.561761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:28.561761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:28.561761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:28.561761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:28.561761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:28.561761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:28.561761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:28.561761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:28.561761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:28.561761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:28.561761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:28.561761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:28.561761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:28.562809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:28.562809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:28.562809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:28.562809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 303/603] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:28.567014 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d53e89-e2e3-4e81-a2e7-759159f36808 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:28.567014 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d53e89-e2e3-4e81-a2e7-759159f36808 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:28.567014 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d53e89-e2e3-4e81-a2e7-759159f36808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:28.591510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-e0d53e89-e2e3-4e81-a2e7-759159f36808 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:28.593761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e0d53e89-e2e3-4e81-a2e7-759159f36808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:28.678284 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:28.699696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d101d3b-db6d-4bdc-b5f1-ba7766eba326 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:28.733004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 304/604] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:28.734370 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d53e89-e2e3-4e81-a2e7-759159f36808 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:28.756493 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f96b0-9331-4e6e-898a-9de05d381a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:28.757927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f96b0-9331-4e6e-898a-9de05d381a0f None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:28.758411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f96b0-9331-4e6e-898a-9de05d381a0f None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:28.758850 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f96b0-9331-4e6e-898a-9de05d381a0f None admin] PATH_INFO: `/v3/projects/4574cf3ff270485faf60594019f7e05c/users/97c363046db444cbb3d4346066f9ad42/roles/743ed2d8e7114975bd9245742a41fe0f` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:28.786619 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:28.793294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f96b0-9331-4e6e-898a-9de05d381a0f None admin] RBAC: Authorizing `identity:create_grant(project_id=4574cf3ff270485faf60594019f7e05c, user_id=97c363046db444cbb3d4346066f9ad42, role_id=743ed2d8e7114975bd9245742a41fe0f)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:28.796462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f96b0-9331-4e6e-898a-9de05d381a0f None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:28.878882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d53e89-e2e3-4e81-a2e7-759159f36808 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:28.931738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104b96e-290b-440c-8fe7-9d5782536c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:28.932458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104b96e-290b-440c-8fe7-9d5782536c01 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:28.932738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104b96e-290b-440c-8fe7-9d5782536c01 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:28.933087 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104b96e-290b-440c-8fe7-9d5782536c01 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:28.933940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104b96e-290b-440c-8fe7-9d5782536c01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:28.934290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104b96e-290b-440c-8fe7-9d5782536c01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:28.936076 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104b96e-290b-440c-8fe7-9d5782536c01 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:28.952841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 305/605] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:28.956470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 301/606] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:09:28 2022] PUT /identity/v3/projects/4574cf3ff270485faf60594019f7e05c/users/97c363046db444cbb3d4346066f9ad42/roles/743ed2d8e7114975bd9245742a41fe0f => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:09:28.972864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104b96e-290b-440c-8fe7-9d5782536c01 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:29.032898 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:29.152068 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104b96e-290b-440c-8fe7-9d5782536c01 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:29.227344 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecc16f-41fa-41d8-b9ba-87272eb089e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:29.228348 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecc16f-41fa-41d8-b9ba-87272eb089e8 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:29.228490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecc16f-41fa-41d8-b9ba-87272eb089e8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:29.228877 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecc16f-41fa-41d8-b9ba-87272eb089e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:29.247844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecc16f-41fa-41d8-b9ba-87272eb089e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:29.255481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecc16f-41fa-41d8-b9ba-87272eb089e8 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:29.287553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-58ecc16f-41fa-41d8-b9ba-87272eb089e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:29.287553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:29.287553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:29.287553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:29.287553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:29.287553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:29.287553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:29.287553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:29.287553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:29.287553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:29.287553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:29.287553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:29.287553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:29.287553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:29.287553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:29.289327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:29.289327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:29.289327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:29.289327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:29.289327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:29.289327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:29.289327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:29.289327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:29.289327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:29.289327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:29.289327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:29.289327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:29.289327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:29.289327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:29.289327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:29.289327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:29.289327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:29.292471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:29.292471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:29.292471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:29.292471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:29.292471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:29.292471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:29.292471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:29.292471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:29.292471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 306/607] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:29.306748 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f96b0-9331-4e6e-898a-9de05d381a0f None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:29.337805 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecc16f-41fa-41d8-b9ba-87272eb089e8 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:29.477917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:29.511279 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecc16f-41fa-41d8-b9ba-87272eb089e8 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:29.521547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f96b0-9331-4e6e-898a-9de05d381a0f None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:29.565965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d005580-2bd3-4744-a26a-b08b6ddc1eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:29.566842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d005580-2bd3-4744-a26a-b08b6ddc1eeb None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:29.567162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d005580-2bd3-4744-a26a-b08b6ddc1eeb None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:29.567553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d005580-2bd3-4744-a26a-b08b6ddc1eeb None admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:29.568745 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d005580-2bd3-4744-a26a-b08b6ddc1eeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:29.569222 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d005580-2bd3-4744-a26a-b08b6ddc1eeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:29.571651 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d005580-2bd3-4744-a26a-b08b6ddc1eeb None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:29.593939 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 302/608] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:29.606749 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d005580-2bd3-4744-a26a-b08b6ddc1eeb None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:29.640340 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602d68b-ef6f-435a-b2bf-e26c4d897b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:29.641230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602d68b-ef6f-435a-b2bf-e26c4d897b70 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:29.641512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602d68b-ef6f-435a-b2bf-e26c4d897b70 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:29.641789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602d68b-ef6f-435a-b2bf-e26c4d897b70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:29.675664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602d68b-ef6f-435a-b2bf-e26c4d897b70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:29.680417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-a602d68b-ef6f-435a-b2bf-e26c4d897b70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:29.680417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:29.680417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:29.680417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:29.680417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:29.680417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:29.680417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:29.680417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:29.680417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:29.680417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:29.680417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:29.680417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:29.680417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:29.680417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:29.680417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:29.680417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:29.682329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:29.682329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:29.682329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:29.682329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:29.682329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:29.682329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:29.682329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:29.682329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:29.682329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:29.682329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:29.682329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:29.682329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:29.682329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:29.682329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:29.682329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:29.682329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:29.684341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:29.684341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:29.684341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:29.684341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:29.684341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:29.684341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:29.684341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:29.684341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:29.684341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:29.684341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:29.684341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:29.684341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:29.684341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 307/609] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:09:29.696720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602d68b-ef6f-435a-b2bf-e26c4d897b70 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:29.720019 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:29.883499 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d005580-2bd3-4744-a26a-b08b6ddc1eeb None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:29.912292 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602d68b-ef6f-435a-b2bf-e26c4d897b70 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:29.985787 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523d15c-f997-4656-a844-edf9a7b6783f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:29.991212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523d15c-f997-4656-a844-edf9a7b6783f None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:29.991212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523d15c-f997-4656-a844-edf9a7b6783f None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:29.991212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523d15c-f997-4656-a844-edf9a7b6783f None admin] PATH_INFO: `/v3/projects/4574cf3ff270485faf60594019f7e05c/users/5acdbdb6f23b4f10bcbab0387ea67ddf/roles/743ed2d8e7114975bd9245742a41fe0f` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:30.006227 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602efe4-896f-4b1e-b4be-eb571aa4867f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:30.007011 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602efe4-896f-4b1e-b4be-eb571aa4867f None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:30.007508 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602efe4-896f-4b1e-b4be-eb571aa4867f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:30.007991 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602efe4-896f-4b1e-b4be-eb571aa4867f None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:30.008541 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523d15c-f997-4656-a844-edf9a7b6783f None admin] RBAC: Authorizing `identity:create_grant(project_id=4574cf3ff270485faf60594019f7e05c, user_id=5acdbdb6f23b4f10bcbab0387ea67ddf, role_id=743ed2d8e7114975bd9245742a41fe0f)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:30.009248 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602efe4-896f-4b1e-b4be-eb571aa4867f None admin] RBAC: Adding query filter params (name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:30.009792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602efe4-896f-4b1e-b4be-eb571aa4867f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:30.010204 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523d15c-f997-4656-a844-edf9a7b6783f None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:30.011995 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602efe4-896f-4b1e-b4be-eb571aa4867f None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:30.027372 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 308/610] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:09:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:30.035270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602efe4-896f-4b1e-b4be-eb571aa4867f None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:30.075574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:30.097553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 303/611] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:09:29 2022] PUT /identity/v3/projects/4574cf3ff270485faf60594019f7e05c/users/5acdbdb6f23b4f10bcbab0387ea67ddf/roles/743ed2d8e7114975bd9245742a41fe0f => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:09:30.122233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5d80f-c1ab-4592-84e9-68f362955fc2 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:30.122646 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5d80f-c1ab-4592-84e9-68f362955fc2 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:30.122800 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5d80f-c1ab-4592-84e9-68f362955fc2 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:30.125023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 304/612] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:09:30.132855 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11972f43-191a-4723-ba16-f13f36612328 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:30.134086 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11972f43-191a-4723-ba16-f13f36612328 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:30.134865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11972f43-191a-4723-ba16-f13f36612328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:30.192780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-11972f43-191a-4723-ba16-f13f36612328 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:30.194600 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-11972f43-191a-4723-ba16-f13f36612328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:30.257722 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602efe4-896f-4b1e-b4be-eb571aa4867f None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:30.349954 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16ee8a-8fa9-40e0-9fba-19f3ad9451e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:30.350612 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16ee8a-8fa9-40e0-9fba-19f3ad9451e0 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:30.350895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16ee8a-8fa9-40e0-9fba-19f3ad9451e0 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:30.351202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16ee8a-8fa9-40e0-9fba-19f3ad9451e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:30.370357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16ee8a-8fa9-40e0-9fba-19f3ad9451e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:30.372013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16ee8a-8fa9-40e0-9fba-19f3ad9451e0 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:30.387842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 305/613] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:30.402808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-5d16ee8a-8fa9-40e0-9fba-19f3ad9451e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:30.402808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:30.402808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:30.402808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:30.402808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:30.402808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:30.402808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:30.402808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:30.402808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:30.402808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:30.402808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:30.402808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:30.402808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:30.402808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:30.402808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:30.402808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:30.404712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:30.404712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:30.404712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:30.404712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:30.404712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:30.404712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:30.404712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:30.404712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:30.404712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:30.404712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:30.404712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:30.404712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:30.404712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:30.404712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:30.404712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:30.404712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:30.404712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:30.408599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:30.408599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:30.408599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:30.408599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 309/614] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:30.416890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11972f43-191a-4723-ba16-f13f36612328 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:30.467799 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:30.606365 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11972f43-191a-4723-ba16-f13f36612328 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:30.690758 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964a192-8014-458a-91bb-87e7380fef19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:30.690758 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964a192-8014-458a-91bb-87e7380fef19 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:30.690758 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964a192-8014-458a-91bb-87e7380fef19 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:30.691589 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964a192-8014-458a-91bb-87e7380fef19 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:30.692523 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964a192-8014-458a-91bb-87e7380fef19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:30.693063 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964a192-8014-458a-91bb-87e7380fef19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:30.695741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964a192-8014-458a-91bb-87e7380fef19 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:30.722741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 306/615] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:30.729353 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16ee8a-8fa9-40e0-9fba-19f3ad9451e0 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:30.808323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964a192-8014-458a-91bb-87e7380fef19 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:30.849581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:30.928174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16ee8a-8fa9-40e0-9fba-19f3ad9451e0 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:30.996113 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d39051-cd4b-4d90-a8a6-2b5da7a4ec34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:30.997120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d39051-cd4b-4d90-a8a6-2b5da7a4ec34 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:30.997614 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d39051-cd4b-4d90-a8a6-2b5da7a4ec34 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:30.998108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d39051-cd4b-4d90-a8a6-2b5da7a4ec34 None admin] PATH_INFO: `/v3/users/swift` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:31.010350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d39051-cd4b-4d90-a8a6-2b5da7a4ec34 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:31.011839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d39051-cd4b-4d90-a8a6-2b5da7a4ec34 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:31.026157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-b9d39051-cd4b-4d90-a8a6-2b5da7a4ec34 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:09:31.026157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:31.026157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:31.026157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:31.026157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:31.026157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:31.026157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:31.026157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:31.026157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:31.026157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:31.026157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:31.026157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:31.026157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:31.026157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:31.026157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:31.026157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:31.028929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:31.030895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:31.030895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:31.030895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:31.030895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:31.030895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:31.030895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:31.030895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:31.030895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:31.030895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:09:31.030895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:31.030895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 310/616] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:09:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:09:31.034699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d39051-cd4b-4d90-a8a6-2b5da7a4ec34 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:31.156981 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:31.171587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964a192-8014-458a-91bb-87e7380fef19 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:31.294140 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de7ae0-0000-4145-bf71-39db44d6a50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:31.294843 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de7ae0-0000-4145-bf71-39db44d6a50f None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:31.295246 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de7ae0-0000-4145-bf71-39db44d6a50f None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:31.295538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de7ae0-0000-4145-bf71-39db44d6a50f None admin] PATH_INFO: `/v3/roles/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:31.324969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d39051-cd4b-4d90-a8a6-2b5da7a4ec34 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:31.336210 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de7ae0-0000-4145-bf71-39db44d6a50f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:31.341790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-e7de7ae0-0000-4145-bf71-39db44d6a50f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:31.341790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:31.341790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:31.341790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:31.341790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:31.341790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:31.341790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:31.341790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:31.341790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:31.341790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:31.341790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:31.341790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:31.341790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:31.341790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:31.341790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:31.341790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:31.343482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:31.343482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:31.343482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:31.343482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:31.343482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:31.343482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:31.343482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:31.343482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:31.343482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:31.343482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:31.343482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:31.343482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:31.343482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:31.343482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:31.343482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:31.343482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:31.345663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:31.345663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:31.345663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:31.345663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:31.345663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:31.345663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:31.345663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:31.345663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:31.345663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:31.345663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:31.345663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:31.345663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:31.351451 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 307/617] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 545 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:09:31.356441 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de7ae0-0000-4145-bf71-39db44d6a50f None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:31.384835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:31.409902 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538fcb2-8b2c-49d0-ba8f-9e7e77591568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:31.411685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538fcb2-8b2c-49d0-ba8f-9e7e77591568 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:31.412500 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538fcb2-8b2c-49d0-ba8f-9e7e77591568 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:31.413268 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538fcb2-8b2c-49d0-ba8f-9e7e77591568 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:31.414965 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538fcb2-8b2c-49d0-ba8f-9e7e77591568 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:31.415833 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538fcb2-8b2c-49d0-ba8f-9e7e77591568 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:31.420044 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538fcb2-8b2c-49d0-ba8f-9e7e77591568 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:31.459118 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 311/618] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Sep 7 13:09:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:09:31.462013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6492d1f-cb91-4d3e-8250-c0fe03168a6e None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:31.462938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6492d1f-cb91-4d3e-8250-c0fe03168a6e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:31.463723 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6492d1f-cb91-4d3e-8250-c0fe03168a6e None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:31.466936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 312/619] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:31.468866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6492d1f-cb91-4d3e-8250-c0fe03168a6e None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:31.611981 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de7ae0-0000-4145-bf71-39db44d6a50f None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:31.672912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6492d1f-cb91-4d3e-8250-c0fe03168a6e None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:31.674577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d42bf1-0ac2-4ead-a29f-85d5399e5c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:31.675671 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d42bf1-0ac2-4ead-a29f-85d5399e5c61 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:31.676353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d42bf1-0ac2-4ead-a29f-85d5399e5c61 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:31.676881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d42bf1-0ac2-4ead-a29f-85d5399e5c61 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:31.678112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d42bf1-0ac2-4ead-a29f-85d5399e5c61 None admin] RBAC: Adding query filter params (name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:31.678696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d42bf1-0ac2-4ead-a29f-85d5399e5c61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:31.681218 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d42bf1-0ac2-4ead-a29f-85d5399e5c61 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:31.696896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 308/620] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:09:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:31.699529 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452c753-c61c-4f50-86a8-bcfaf5c2dcb4 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:31.700071 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452c753-c61c-4f50-86a8-bcfaf5c2dcb4 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:31.700606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452c753-c61c-4f50-86a8-bcfaf5c2dcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:31.732269 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-5452c753-c61c-4f50-86a8-bcfaf5c2dcb4 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:31.734786 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5452c753-c61c-4f50-86a8-bcfaf5c2dcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:31.742796 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a1ac7-688d-45a9-aaf8-cd392fb69459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:31.744357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a1ac7-688d-45a9-aaf8-cd392fb69459 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:31.745395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a1ac7-688d-45a9-aaf8-cd392fb69459 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:31.746175 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a1ac7-688d-45a9-aaf8-cd392fb69459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:31.755139 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:31.769001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a1ac7-688d-45a9-aaf8-cd392fb69459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:31.772393 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a1ac7-688d-45a9-aaf8-cd392fb69459 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:31.802703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-da8a1ac7-688d-45a9-aaf8-cd392fb69459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:31.802703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:31.802703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:31.802703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:31.802703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:31.802703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:31.802703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:31.802703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:31.802703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:31.802703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:31.802703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:31.802703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:31.802703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:31.802703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:31.802703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:31.802703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:31.804688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:31.804688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:31.804688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:31.804688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:31.804688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:31.804688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:31.804688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:31.804688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:31.804688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:31.804688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:31.804688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:31.804688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:31.804688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:31.804688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:31.804688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:31.804688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:31.804688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:31.806439 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:31.806439 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:31.806439 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:31.810832 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 313/621] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:31.812698 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a1ac7-688d-45a9-aaf8-cd392fb69459 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:31.868741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 309/622] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:31.870415 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452c753-c61c-4f50-86a8-bcfaf5c2dcb4 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:31.880562 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:32.007782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a1ac7-688d-45a9-aaf8-cd392fb69459 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:32.059242 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452c753-c61c-4f50-86a8-bcfaf5c2dcb4 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:32.071555 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730253d5-5634-489e-87c9-916ef83c8e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:32.073048 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730253d5-5634-489e-87c9-916ef83c8e12 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:32.073842 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730253d5-5634-489e-87c9-916ef83c8e12 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:32.074717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730253d5-5634-489e-87c9-916ef83c8e12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:32.095837 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730253d5-5634-489e-87c9-916ef83c8e12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:32.098999 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730253d5-5634-489e-87c9-916ef83c8e12 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:32.130841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-730253d5-5634-489e-87c9-916ef83c8e12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:32.130841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:32.130841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:32.130841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:32.130841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:32.130841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:32.130841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:32.130841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:32.130841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:32.130841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:32.130841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:32.130841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:32.130841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:32.130841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:32.130841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:32.130841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:32.132655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:32.132655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:32.132655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:32.132655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:32.132655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:32.132655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:32.132655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:32.132655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:32.132655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:32.132655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:32.132655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:32.132655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:32.132655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:32.132655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:32.132655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:32.132655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:32.132655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:32.134457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:32.134457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:32.134457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:32.136602 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51447458-fed5-48e2-9bfd-f114a4849b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:32.137628 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51447458-fed5-48e2-9bfd-f114a4849b31 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:32.138105 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51447458-fed5-48e2-9bfd-f114a4849b31 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:32.138629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51447458-fed5-48e2-9bfd-f114a4849b31 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:32.139937 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51447458-fed5-48e2-9bfd-f114a4849b31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:32.140270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 314/623] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:32.142010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51447458-fed5-48e2-9bfd-f114a4849b31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:32.150733 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51447458-fed5-48e2-9bfd-f114a4849b31 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:32.153789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730253d5-5634-489e-87c9-916ef83c8e12 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:32.179382 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 310/624] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:32.189432 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51447458-fed5-48e2-9bfd-f114a4849b31 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:32.231653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:32.379872 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:32.405441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730253d5-5634-489e-87c9-916ef83c8e12 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:32.472372 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7557102-de40-4251-bcf9-be95e90d019d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:32.473084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7557102-de40-4251-bcf9-be95e90d019d None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:32.473375 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7557102-de40-4251-bcf9-be95e90d019d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:32.473657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7557102-de40-4251-bcf9-be95e90d019d None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:32.475251 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7557102-de40-4251-bcf9-be95e90d019d None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:32.475568 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7557102-de40-4251-bcf9-be95e90d019d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:32.477422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7557102-de40-4251-bcf9-be95e90d019d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:32.507932 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51447458-fed5-48e2-9bfd-f114a4849b31 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:32.511521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 315/625] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:32.514735 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7557102-de40-4251-bcf9-be95e90d019d None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:32.592635 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5ebd4-5f60-402a-a484-5054d02c864e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:32.593784 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5ebd4-5f60-402a-a484-5054d02c864e None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:32.596545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5ebd4-5f60-402a-a484-5054d02c864e None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:32.597081 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5ebd4-5f60-402a-a484-5054d02c864e None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:32.616004 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5ebd4-5f60-402a-a484-5054d02c864e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:32.619644 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5ebd4-5f60-402a-a484-5054d02c864e None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:32.646725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-a9d5ebd4-5f60-402a-a484-5054d02c864e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:32.646725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:32.646725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:32.646725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:32.646725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:32.646725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:32.646725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:32.646725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:32.646725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:32.646725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:32.646725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:32.646725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:32.646725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:32.646725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:32.646725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:32.648577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:32.648577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:32.648577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:32.648577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:32.648577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:32.648577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:32.648577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:32.648577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:32.648577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:32.648577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:32.648577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:32.648577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:32.648577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:32.648577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:32.648577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:32.648577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:32.648577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:32.650339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:32.650339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:32.650339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:32.650339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:32.650339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:32.650339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:32.650339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:32.650339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:32.655157 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 311/626] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 467 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:32.658077 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5ebd4-5f60-402a-a484-5054d02c864e None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:32.692265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7557102-de40-4251-bcf9-be95e90d019d None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:32.753380 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d1fc0-70b8-4259-bf22-48784f09ca92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:32.754768 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d1fc0-70b8-4259-bf22-48784f09ca92 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:32.755120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d1fc0-70b8-4259-bf22-48784f09ca92 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:32.755407 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d1fc0-70b8-4259-bf22-48784f09ca92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:32.766613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:32.788625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888d1fc0-70b8-4259-bf22-48784f09ca92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:32.805451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-888d1fc0-70b8-4259-bf22-48784f09ca92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:32.805451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:32.805451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:32.805451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:32.805451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:32.805451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:32.805451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:32.805451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:32.805451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:32.805451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:32.805451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:32.805451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:32.805451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:32.805451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:32.805451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:32.805451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:32.806867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:32.806867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:32.806867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:32.806867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:32.806867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:32.806867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:32.806867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:32.806867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:32.806867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:32.806867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:32.806867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:32.806867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:32.806867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:32.806867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:32.806867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:32.806867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:32.808382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:32.808382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:32.808382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:32.808382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:32.808382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:32.808382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:32.808382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:32.808382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:32.808382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:32.808382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:32.808382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:32.808382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:32.809056 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 316/627] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:32.815998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d1fc0-70b8-4259-bf22-48784f09ca92 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:32.828220 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:32.871247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5ebd4-5f60-402a-a484-5054d02c864e None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:32.944726 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c1b01-37ad-4cde-ad35-3342d6e4b824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:32.945805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c1b01-37ad-4cde-ad35-3342d6e4b824 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:32.946296 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c1b01-37ad-4cde-ad35-3342d6e4b824 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:32.946808 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c1b01-37ad-4cde-ad35-3342d6e4b824 None admin] PATH_INFO: `/v3/users/glance` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:32.957002 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c1b01-37ad-4cde-ad35-3342d6e4b824 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:32.958479 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c1b01-37ad-4cde-ad35-3342d6e4b824 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:32.971630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-f61c1b01-37ad-4cde-ad35-3342d6e4b824 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 13:09:32.971630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:32.971630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:32.971630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:32.971630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:32.971630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:32.971630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:32.971630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:32.971630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:32.971630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:32.971630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:32.971630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:32.971630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:32.971630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:32.971630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:32.971630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:32.972980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:32.974033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:32.974033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:32.974033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:32.974033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:32.974033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:32.974033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:32.974033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:32.974033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:32.974033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 13:09:32.974033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:32.974033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 312/628] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:09:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:32.974659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c1b01-37ad-4cde-ad35-3342d6e4b824 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:33.052822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d1fc0-70b8-4259-bf22-48784f09ca92 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:33.128217 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58e85f-a90e-408c-baa9-40dcd3ee437d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:33.129290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58e85f-a90e-408c-baa9-40dcd3ee437d None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:33.129780 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58e85f-a90e-408c-baa9-40dcd3ee437d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:33.130268 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58e85f-a90e-408c-baa9-40dcd3ee437d None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:33.131629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58e85f-a90e-408c-baa9-40dcd3ee437d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:33.132374 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58e85f-a90e-408c-baa9-40dcd3ee437d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:33.135184 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58e85f-a90e-408c-baa9-40dcd3ee437d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:33.178917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 317/629] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:33.181407 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c25bc-1f02-4646-acde-f0ae5447f8a1 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:33.181687 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c1b01-37ad-4cde-ad35-3342d6e4b824 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:33.181906 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c25bc-1f02-4646-acde-f0ae5447f8a1 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:33.182398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c25bc-1f02-4646-acde-f0ae5447f8a1 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:33.191040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 318/630] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:33.191040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c25bc-1f02-4646-acde-f0ae5447f8a1 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:33.242666 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:33.268541 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b22e85-ad1f-44cd-9510-731475333c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:33.276394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b22e85-ad1f-44cd-9510-731475333c91 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:33.276394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b22e85-ad1f-44cd-9510-731475333c91 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:33.276394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b22e85-ad1f-44cd-9510-731475333c91 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:33.276394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b22e85-ad1f-44cd-9510-731475333c91 None admin] RBAC: Adding query filter params (name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:33.276394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b22e85-ad1f-44cd-9510-731475333c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:33.276394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b22e85-ad1f-44cd-9510-731475333c91 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:33.294027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 313/631] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:09:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:33.299137 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b22e85-ad1f-44cd-9510-731475333c91 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:33.343379 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:33.417424 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c25bc-1f02-4646-acde-f0ae5447f8a1 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:33.492206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0de536-b520-4edd-86c9-085c7f34b1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:33.493076 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0de536-b520-4edd-86c9-085c7f34b1a0 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:33.493342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0de536-b520-4edd-86c9-085c7f34b1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:33.493613 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0de536-b520-4edd-86c9-085c7f34b1a0 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:33.494638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b22e85-ad1f-44cd-9510-731475333c91 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:33.494842 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0de536-b520-4edd-86c9-085c7f34b1a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:33.495166 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0de536-b520-4edd-86c9-085c7f34b1a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:33.496841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0de536-b520-4edd-86c9-085c7f34b1a0 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:33.528085 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 319/632] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Sep 7 13:09:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:33.529786 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6991e-c75c-4364-88dc-669d3438d390 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:33.529786 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6991e-c75c-4364-88dc-669d3438d390 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:33.529786 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6991e-c75c-4364-88dc-669d3438d390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:33.564941 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-47f6991e-c75c-4364-88dc-669d3438d390 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:33.567304 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-47f6991e-c75c-4364-88dc-669d3438d390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:33.578794 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d790008-b31f-4237-891f-c6a1908d2c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:33.579463 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d790008-b31f-4237-891f-c6a1908d2c29 None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:33.579907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d790008-b31f-4237-891f-c6a1908d2c29 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:33.580387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d790008-b31f-4237-891f-c6a1908d2c29 None admin] PATH_INFO: `/v3/projects/4574cf3ff270485faf60594019f7e05c/users/5cedc7091d2f46b5be68818d26399e9b/roles/743ed2d8e7114975bd9245742a41fe0f` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:33.606191 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d790008-b31f-4237-891f-c6a1908d2c29 None admin] RBAC: Authorizing `identity:create_grant(project_id=4574cf3ff270485faf60594019f7e05c, user_id=5cedc7091d2f46b5be68818d26399e9b, role_id=743ed2d8e7114975bd9245742a41fe0f)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:33.608996 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d790008-b31f-4237-891f-c6a1908d2c29 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:33.697885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 314/633] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:09:33 2022] PUT /identity/v3/projects/4574cf3ff270485faf60594019f7e05c/users/5cedc7091d2f46b5be68818d26399e9b/roles/743ed2d8e7114975bd9245742a41fe0f => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:09:33.700733 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d790008-b31f-4237-891f-c6a1908d2c29 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:33.702034 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 320/634] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:33.704205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6991e-c75c-4364-88dc-669d3438d390 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:33.774843 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:33.827891 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:33.999793 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6991e-c75c-4364-88dc-669d3438d390 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:34.028275 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d790008-b31f-4237-891f-c6a1908d2c29 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:34.120888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf8cb3-6a6e-4f3c-9c8d-4d03a92f8cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:34.121922 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf8cb3-6a6e-4f3c-9c8d-4d03a92f8cef None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:34.122367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf8cb3-6a6e-4f3c-9c8d-4d03a92f8cef None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:34.122799 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf8cb3-6a6e-4f3c-9c8d-4d03a92f8cef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:34.134737 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd020b-7ffc-4853-abfb-af954d4ab2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:34.135867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd020b-7ffc-4853-abfb-af954d4ab2c6 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:34.136354 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd020b-7ffc-4853-abfb-af954d4ab2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:34.136829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd020b-7ffc-4853-abfb-af954d4ab2c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:34.144828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cf8cb3-6a6e-4f3c-9c8d-4d03a92f8cef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:34.147412 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cf8cb3-6a6e-4f3c-9c8d-4d03a92f8cef None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:34.177045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-a5cf8cb3-6a6e-4f3c-9c8d-4d03a92f8cef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:34.177045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:34.177045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:34.177045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:34.177045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:34.177045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:34.177045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:34.177045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:34.177045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:34.177045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:34.177045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:34.177045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:34.177045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:34.177045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:34.177045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:34.177045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:34.178118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:34.178118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:34.178118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:34.178118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:34.178118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:34.178118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:34.178118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:34.178118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:34.178118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:34.178118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:34.178118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:34.178118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:34.178118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:34.178118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:34.178118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:34.178118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:34.178118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:34.180388 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:34.180388 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:34.180388 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:34.180388 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 315/635] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 479 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:34.189178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf8cb3-6a6e-4f3c-9c8d-4d03a92f8cef None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:34.300362 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd020b-7ffc-4853-abfb-af954d4ab2c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:34.303112 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd020b-7ffc-4853-abfb-af954d4ab2c6 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:34.384680 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:34.408992 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf8cb3-6a6e-4f3c-9c8d-4d03a92f8cef None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:34.420606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:34.475008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398fa38-ccee-4e55-a06f-5afad4b9d5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:34.475674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398fa38-ccee-4e55-a06f-5afad4b9d5e9 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:34.478773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398fa38-ccee-4e55-a06f-5afad4b9d5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:34.478773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398fa38-ccee-4e55-a06f-5afad4b9d5e9 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:34.478773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398fa38-ccee-4e55-a06f-5afad4b9d5e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:34.478773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398fa38-ccee-4e55-a06f-5afad4b9d5e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:34.478773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398fa38-ccee-4e55-a06f-5afad4b9d5e9 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:34.495925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 321/636] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:09:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 793 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:09:34.497291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd020b-7ffc-4853-abfb-af954d4ab2c6 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:34.510715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 316/637] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:09:34.518570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9733d-ad62-4437-ba5e-0c713d184812 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:34.519102 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9733d-ad62-4437-ba5e-0c713d184812 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:34.519598 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9733d-ad62-4437-ba5e-0c713d184812 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:34.527629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 317/638] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:34 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:34.529097 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9733d-ad62-4437-ba5e-0c713d184812 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:34.680973 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd020b-7ffc-4853-abfb-af954d4ab2c6 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:34.739189 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fa120-48ec-4a64-8c91-e8b283956fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:34.739881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fa120-48ec-4a64-8c91-e8b283956fef None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:34.740594 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fa120-48ec-4a64-8c91-e8b283956fef None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:34.740921 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fa120-48ec-4a64-8c91-e8b283956fef None admin] PATH_INFO: `/v3/roles/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:34.744254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9733d-ad62-4437-ba5e-0c713d184812 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:34.758862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fa120-48ec-4a64-8c91-e8b283956fef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:34.765557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-a57fa120-48ec-4a64-8c91-e8b283956fef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:34.765557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:34.765557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:34.765557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:34.765557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:34.765557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:34.765557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:34.765557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:34.765557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:34.765557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:34.765557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:34.765557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:34.765557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:34.765557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:34.765557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:34.765557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:34.768001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:34.768001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:34.768001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:34.768001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:34.768001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:34.768001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:34.768001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:34.768001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:34.768001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:34.768001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:34.768001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:34.768001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:34.768001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:34.768001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:34.768001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:34.768001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:34.769802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:34.769802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:34.769802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:34.769802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:34.769802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:34.769802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:34.769802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:34.769802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:34.769802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:34.769802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:34.769802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:34.769802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:34.769802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 322/639] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:34.771210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fa120-48ec-4a64-8c91-e8b283956fef None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:34.796580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:34.804308 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3e73e-c5e9-4840-bea2-690f1d3e6e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:34.805270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3e73e-c5e9-4840-bea2-690f1d3e6e0a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:34.805715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3e73e-c5e9-4840-bea2-690f1d3e6e0a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:34.806135 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3e73e-c5e9-4840-bea2-690f1d3e6e0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:34.825549 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3e73e-c5e9-4840-bea2-690f1d3e6e0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:34.829235 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3e73e-c5e9-4840-bea2-690f1d3e6e0a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:34.846317 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:34.860838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-add3e73e-c5e9-4840-bea2-690f1d3e6e0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:34.860838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:34.860838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:34.860838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:34.860838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:34.860838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:34.860838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:34.860838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:34.860838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:34.860838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:34.860838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:34.860838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:34.860838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:34.860838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:34.860838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:34.860838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:34.861999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:34.861999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:34.861999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:34.861999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:34.861999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:34.861999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:34.861999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:34.861999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:34.861999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:34.861999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:34.861999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:34.861999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:34.861999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:34.861999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:34.861999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:34.861999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:34.861999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:34.863827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:34.863827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:34.863827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:34.863827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 318/640] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:34.865271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4d31f-e0de-4b78-9a0e-1e78e364119a None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:34.865762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4d31f-e0de-4b78-9a0e-1e78e364119a None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:34.866236 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4d31f-e0de-4b78-9a0e-1e78e364119a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:34.894093 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-b9b4d31f-e0de-4b78-9a0e-1e78e364119a None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:34.895503 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b9b4d31f-e0de-4b78-9a0e-1e78e364119a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:34.961961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fa120-48ec-4a64-8c91-e8b283956fef None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:35.017453 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43063564-139f-467d-bd44-5815a07f247c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:35.018355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 319/641] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:35.019394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4d31f-e0de-4b78-9a0e-1e78e364119a None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:35.027615 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43063564-139f-467d-bd44-5815a07f247c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:35.027929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43063564-139f-467d-bd44-5815a07f247c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:35.028272 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43063564-139f-467d-bd44-5815a07f247c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:35.053618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43063564-139f-467d-bd44-5815a07f247c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:35.056751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43063564-139f-467d-bd44-5815a07f247c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:35.080690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-43063564-139f-467d-bd44-5815a07f247c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 13:09:35.080690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:35.080690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:35.080690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:35.080690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:35.080690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:35.080690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:35.080690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:35.080690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:35.080690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:35.080690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:35.080690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:35.080690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:35.080690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:35.080690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:35.080690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:35.083225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:35.085427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:35.085427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:35.085427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:35.085427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:35.085427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:35.085427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:35.085427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:35.085427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:35.085427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 13:09:35.085427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:35.091149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 323/642] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:09:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:35.092951 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43063564-139f-467d-bd44-5815a07f247c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:35.200682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4d31f-e0de-4b78-9a0e-1e78e364119a None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:35.244348 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:35.251489 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:35.261239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784cdb0-c2a4-47ae-93db-0e847c1b3057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:35.262648 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784cdb0-c2a4-47ae-93db-0e847c1b3057 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:35.263000 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784cdb0-c2a4-47ae-93db-0e847c1b3057 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:35.263786 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784cdb0-c2a4-47ae-93db-0e847c1b3057 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:35.265198 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0784cdb0-c2a4-47ae-93db-0e847c1b3057 None admin] RBAC: Adding query filter params (name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:35.265759 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0784cdb0-c2a4-47ae-93db-0e847c1b3057 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:35.269089 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0784cdb0-c2a4-47ae-93db-0e847c1b3057 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:35.291102 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 320/643] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:09:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:35.292823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784cdb0-c2a4-47ae-93db-0e847c1b3057 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:35.303864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43063564-139f-467d-bd44-5815a07f247c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:35.371897 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2614e-c0a2-46d4-a7b3-2bd0b4c5335a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:35.373481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2614e-c0a2-46d4-a7b3-2bd0b4c5335a None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:35.374153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2614e-c0a2-46d4-a7b3-2bd0b4c5335a None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:35.374794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2614e-c0a2-46d4-a7b3-2bd0b4c5335a None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:35.376860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2614e-c0a2-46d4-a7b3-2bd0b4c5335a None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:35.377578 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2614e-c0a2-46d4-a7b3-2bd0b4c5335a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:35.380326 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2614e-c0a2-46d4-a7b3-2bd0b4c5335a None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:35.404919 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 324/644] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:35.407552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2614e-c0a2-46d4-a7b3-2bd0b4c5335a None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:35.476119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784cdb0-c2a4-47ae-93db-0e847c1b3057 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:35.537672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87698e10-a26e-41bc-aaae-8286f9889404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:35.539058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87698e10-a26e-41bc-aaae-8286f9889404 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:35.539748 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87698e10-a26e-41bc-aaae-8286f9889404 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:35.540413 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87698e10-a26e-41bc-aaae-8286f9889404 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:35.541795 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87698e10-a26e-41bc-aaae-8286f9889404 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:35.542461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87698e10-a26e-41bc-aaae-8286f9889404 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:35.544977 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87698e10-a26e-41bc-aaae-8286f9889404 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:35.563869 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2614e-c0a2-46d4-a7b3-2bd0b4c5335a None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:35.588756 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 321/645] 10.222.0.62 () {56 vars in 1137 bytes} [Wed Sep 7 13:09:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:35.588756 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87698e10-a26e-41bc-aaae-8286f9889404 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:35.620205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6ec5c-b910-465b-b643-1f83d1840895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:35.621381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6ec5c-b910-465b-b643-1f83d1840895 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:35.621956 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6ec5c-b910-465b-b643-1f83d1840895 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:35.622519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6ec5c-b910-465b-b643-1f83d1840895 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:35.627210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:35.643116 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6ec5c-b910-465b-b643-1f83d1840895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:35.644753 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6ec5c-b910-465b-b643-1f83d1840895 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:35.662699 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:35.663747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-71c6ec5c-b910-465b-b643-1f83d1840895 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:35.663747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:35.663747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:35.663747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:35.663747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:35.663747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:35.663747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:35.663747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:35.663747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:35.663747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:35.663747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:35.663747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:35.663747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:35.663747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:35.663747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:35.663747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:35.665935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:35.665935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:35.665935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:35.665935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:35.665935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:35.665935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:35.665935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:35.665935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:35.665935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:35.665935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:35.665935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:35.665935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:35.665935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:35.665935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:35.665935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:35.665935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:35.665935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:35.670896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:35.670896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:35.670896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:35.670896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 325/646] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:35.671888 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6ec5c-b910-465b-b643-1f83d1840895 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:35.744153 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87698e10-a26e-41bc-aaae-8286f9889404 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:35.815623 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37248a-71c3-4ec1-b7ee-f06778ccadc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:35.816548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37248a-71c3-4ec1-b7ee-f06778ccadc4 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:35.817025 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37248a-71c3-4ec1-b7ee-f06778ccadc4 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:35.817491 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37248a-71c3-4ec1-b7ee-f06778ccadc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:35.817681 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6ec5c-b910-465b-b643-1f83d1840895 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:35.832272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b37248a-71c3-4ec1-b7ee-f06778ccadc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:35.835139 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b37248a-71c3-4ec1-b7ee-f06778ccadc4 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:35.859151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-7b37248a-71c3-4ec1-b7ee-f06778ccadc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:35.859151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:35.859151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:35.859151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:35.859151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:35.859151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:35.859151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:35.859151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:35.859151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:35.859151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:35.859151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:35.859151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:35.859151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:35.859151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:35.859151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:35.860957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:35.860957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:35.860957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:35.860957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:35.860957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:35.860957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:35.860957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:35.860957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:35.860957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:35.860957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:35.860957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:35.860957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:35.860957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:35.860957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:35.860957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:35.860957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:35.860957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:35.861901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:35.861901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:35.861901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:35.861901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:35.861901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:35.861901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:35.861901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:35.861901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:35.863146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 322/647] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:35.864195 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37248a-71c3-4ec1-b7ee-f06778ccadc4 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:35.876115 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c95376-2806-4369-a7d9-2c12fff3ac87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:35.877157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c95376-2806-4369-a7d9-2c12fff3ac87 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:35.877630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c95376-2806-4369-a7d9-2c12fff3ac87 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:35.878078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c95376-2806-4369-a7d9-2c12fff3ac87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:35.894045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c95376-2806-4369-a7d9-2c12fff3ac87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:35.904469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-a4c95376-2806-4369-a7d9-2c12fff3ac87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:35.904469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:35.904469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:35.904469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:35.904469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:35.904469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:35.904469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:35.904469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:35.904469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:35.904469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:35.904469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:35.904469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:35.904469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:35.904469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:35.904469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:35.904469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:35.905575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:35.905575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:35.905575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:35.905575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:35.905575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:35.905575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:35.905575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:35.905575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:35.905575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:35.905575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:35.905575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:35.905575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:35.905575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:35.905575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:35.905575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:35.905575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:35.907000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:35.907000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:35.907000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:35.907000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:35.907000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:35.907000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:35.907000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:35.907000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:35.907000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:35.907000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:35.907000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:35.907000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:35.908049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 326/648] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:35.909549 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c95376-2806-4369-a7d9-2c12fff3ac87 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:35.985119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:36.047009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37248a-71c3-4ec1-b7ee-f06778ccadc4 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:36.066122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:36.084953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c95376-2806-4369-a7d9-2c12fff3ac87 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:36.098189 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578055e-06ed-4190-a223-7e98a5c0ce52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:36.098890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578055e-06ed-4190-a223-7e98a5c0ce52 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:36.099285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578055e-06ed-4190-a223-7e98a5c0ce52 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:36.099492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578055e-06ed-4190-a223-7e98a5c0ce52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:36.136927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d8b28-e725-47fe-9a2b-84af54b56534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:36.137709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d8b28-e725-47fe-9a2b-84af54b56534 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:36.137969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d8b28-e725-47fe-9a2b-84af54b56534 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:36.138241 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d8b28-e725-47fe-9a2b-84af54b56534 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:36.138954 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d8b28-e725-47fe-9a2b-84af54b56534 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:36.139295 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d8b28-e725-47fe-9a2b-84af54b56534 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:36.142186 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d8b28-e725-47fe-9a2b-84af54b56534 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:36.163310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 327/649] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:36.163310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d8b28-e725-47fe-9a2b-84af54b56534 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:36.222108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578055e-06ed-4190-a223-7e98a5c0ce52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:36.223793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578055e-06ed-4190-a223-7e98a5c0ce52 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:36.320927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d8b28-e725-47fe-9a2b-84af54b56534 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:36.331554 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:36.374631 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52194ef0-0c13-4cf2-aebe-8230d322a8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:36.377321 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52194ef0-0c13-4cf2-aebe-8230d322a8f9 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:36.377672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52194ef0-0c13-4cf2-aebe-8230d322a8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:36.378015 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52194ef0-0c13-4cf2-aebe-8230d322a8f9 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:36.378938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52194ef0-0c13-4cf2-aebe-8230d322a8f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:36.380171 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52194ef0-0c13-4cf2-aebe-8230d322a8f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:36.382939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52194ef0-0c13-4cf2-aebe-8230d322a8f9 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:36.406880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 323/650] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:09:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:09:36.406880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578055e-06ed-4190-a223-7e98a5c0ce52 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:36.412488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 328/651] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:36.414077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52194ef0-0c13-4cf2-aebe-8230d322a8f9 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:36.426564 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:36.589240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578055e-06ed-4190-a223-7e98a5c0ce52 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:36.596609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52194ef0-0c13-4cf2-aebe-8230d322a8f9 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:36.663307 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88596e75-976f-46dc-8535-65e79281235d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:36.663933 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88596e75-976f-46dc-8535-65e79281235d None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:36.664212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88596e75-976f-46dc-8535-65e79281235d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:36.664512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88596e75-976f-46dc-8535-65e79281235d None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:36.665297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88596e75-976f-46dc-8535-65e79281235d None admin] RBAC: Adding query filter params (name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:36.665642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88596e75-976f-46dc-8535-65e79281235d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:36.667212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88596e75-976f-46dc-8535-65e79281235d None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:36.669169 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4dcbb-1dd8-4708-a040-eccf4ed787aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:36.670307 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4dcbb-1dd8-4708-a040-eccf4ed787aa None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:36.670801 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4dcbb-1dd8-4708-a040-eccf4ed787aa None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:36.671345 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4dcbb-1dd8-4708-a040-eccf4ed787aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:36.678138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 324/652] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:09:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:36.678657 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88596e75-976f-46dc-8535-65e79281235d None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:36.696067 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4dcbb-1dd8-4708-a040-eccf4ed787aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:36.698700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4dcbb-1dd8-4708-a040-eccf4ed787aa None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:36.723648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-32b4dcbb-1dd8-4708-a040-eccf4ed787aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:09:36.723648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:36.723648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:36.723648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:36.723648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:36.723648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:36.723648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:36.723648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:36.723648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:36.723648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:36.723648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:36.723648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:36.723648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:36.723648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:36.723648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:36.723648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:36.725625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:36.730024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:36.730024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:36.730024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:36.730024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:36.730024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:36.730024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:36.730024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:36.730024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:36.730024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:09:36.730024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:36.730024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 329/653] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 7 13:09:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:36.730024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4dcbb-1dd8-4708-a040-eccf4ed787aa None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:36.780179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:36.790703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:36.846620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88596e75-976f-46dc-8535-65e79281235d None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:36.914478 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cca460-33b5-46b2-9019-db515ac9f9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:36.915641 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cca460-33b5-46b2-9019-db515ac9f9d7 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:36.916135 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cca460-33b5-46b2-9019-db515ac9f9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:36.917024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cca460-33b5-46b2-9019-db515ac9f9d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:36.940626 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cca460-33b5-46b2-9019-db515ac9f9d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:36.941388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4dcbb-1dd8-4708-a040-eccf4ed787aa None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:36.943648 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cca460-33b5-46b2-9019-db515ac9f9d7 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:36.976387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-f5cca460-33b5-46b2-9019-db515ac9f9d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:36.976387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:36.976387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:36.976387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:36.976387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:36.976387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:36.976387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:36.976387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:36.976387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:36.976387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:36.976387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:36.976387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:36.976387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:36.976387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:36.976387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:36.976387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:36.978449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:36.978449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:36.978449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:36.978449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:36.978449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:36.978449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:36.978449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:36.978449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:36.978449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:36.978449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:36.978449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:36.978449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:36.978449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:36.978449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:36.978449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:36.978449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:36.978449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:36.980254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:36.980254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:36.980254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:36.986640 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 325/654] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:36.988292 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cca460-33b5-46b2-9019-db515ac9f9d7 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:37.020510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b8801-fd69-402f-907f-7ff8d7a85531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:37.021963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b8801-fd69-402f-907f-7ff8d7a85531 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:37.022729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b8801-fd69-402f-907f-7ff8d7a85531 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:37.023467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b8801-fd69-402f-907f-7ff8d7a85531 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:37.025166 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b8801-fd69-402f-907f-7ff8d7a85531 None admin] RBAC: Adding query filter params (role.id=743ed2d8e7114975bd9245742a41fe0f, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=d1a48805d95640489d7ce3333dd687b9) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:37.025939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b8801-fd69-402f-907f-7ff8d7a85531 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:37.029100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b8801-fd69-402f-907f-7ff8d7a85531 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:37.042428 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 330/655] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:09:36 2022] GET /identity/v3/role_assignments?user.id=d1a48805d95640489d7ce3333dd687b9&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=743ed2d8e7114975bd9245742a41fe0f => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:37.044381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b8801-fd69-402f-907f-7ff8d7a85531 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:37.188901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cca460-33b5-46b2-9019-db515ac9f9d7 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:37.239331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:37.239549 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:37.253617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b8801-fd69-402f-907f-7ff8d7a85531 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:37.265307 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546a636-623e-4eb4-8494-e10db2f27826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:37.276633 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546a636-623e-4eb4-8494-e10db2f27826 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:37.276633 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546a636-623e-4eb4-8494-e10db2f27826 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:37.276633 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546a636-623e-4eb4-8494-e10db2f27826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:37.290630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546a636-623e-4eb4-8494-e10db2f27826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:37.294024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546a636-623e-4eb4-8494-e10db2f27826 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:37.314941 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3cfd3-9957-4ac6-88c4-85fe4a0684dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:37.315785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-2546a636-623e-4eb4-8494-e10db2f27826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:37.315785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:37.315785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:37.315785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:37.315785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:37.315785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:37.315785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:37.315785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:37.315785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:37.315785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:37.315785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:37.315785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:37.315785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:37.315785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:37.315785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:37.315785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:37.335647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:37.335647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:37.335647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:37.335647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:37.335647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:37.335647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:37.335647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:37.335647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:37.335647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:37.335647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:37.335647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:37.335647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:37.335647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:37.335647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:37.335647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:37.335647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:37.335647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:37.337802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:37.337802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:37.337802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:37.337996 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3cfd3-9957-4ac6-88c4-85fe4a0684dd None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:37.337996 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3cfd3-9957-4ac6-88c4-85fe4a0684dd None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:37.338152 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 326/656] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:37.338225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3cfd3-9957-4ac6-88c4-85fe4a0684dd None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:37.338344 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546a636-623e-4eb4-8494-e10db2f27826 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:37.338437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3cfd3-9957-4ac6-88c4-85fe4a0684dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:37.338437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3cfd3-9957-4ac6-88c4-85fe4a0684dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:37.338437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3cfd3-9957-4ac6-88c4-85fe4a0684dd None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:37.361520 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 331/657] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 7 13:09:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:37.362995 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f90fdf-86e6-4a99-b585-98c2e403e109 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:37.363353 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f90fdf-86e6-4a99-b585-98c2e403e109 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:37.363774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f90fdf-86e6-4a99-b585-98c2e403e109 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:37.365396 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 332/658] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:37.368130 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f90fdf-86e6-4a99-b585-98c2e403e109 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:37.509348 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546a636-623e-4eb4-8494-e10db2f27826 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:37.552069 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f90fdf-86e6-4a99-b585-98c2e403e109 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:37.575779 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc856a-d58c-4252-aa94-e6d2051dd1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:37.576826 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc856a-d58c-4252-aa94-e6d2051dd1ad None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:37.577308 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc856a-d58c-4252-aa94-e6d2051dd1ad None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:37.577777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc856a-d58c-4252-aa94-e6d2051dd1ad None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:37.580495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc856a-d58c-4252-aa94-e6d2051dd1ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:37.581293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc856a-d58c-4252-aa94-e6d2051dd1ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:37.583901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc856a-d58c-4252-aa94-e6d2051dd1ad None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:37.600471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd882b1d-3a7a-4340-a74f-47fe85b19777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:37.601233 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd882b1d-3a7a-4340-a74f-47fe85b19777 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:37.601373 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd882b1d-3a7a-4340-a74f-47fe85b19777 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:37.601664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd882b1d-3a7a-4340-a74f-47fe85b19777 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:37.602316 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd882b1d-3a7a-4340-a74f-47fe85b19777 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:37.602619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd882b1d-3a7a-4340-a74f-47fe85b19777 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:37.604032 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd882b1d-3a7a-4340-a74f-47fe85b19777 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:37.613877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 327/659] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:37.613877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc856a-d58c-4252-aa94-e6d2051dd1ad None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:37.624661 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 333/660] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:37.626775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc3355-b7e7-4601-bb3b-e6de8f8fb7f5 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:37.628234 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc3355-b7e7-4601-bb3b-e6de8f8fb7f5 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:37.628843 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc3355-b7e7-4601-bb3b-e6de8f8fb7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:37.659527 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-83fc3355-b7e7-4601-bb3b-e6de8f8fb7f5 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:37.661637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-83fc3355-b7e7-4601-bb3b-e6de8f8fb7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:37.665573 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:37.673194 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:37.747065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc856a-d58c-4252-aa94-e6d2051dd1ad None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:37.778462 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 334/661] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:37.780193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc3355-b7e7-4601-bb3b-e6de8f8fb7f5 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:37.821497 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b10fa-421b-4493-8f48-edaffa030187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:37.823092 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b10fa-421b-4493-8f48-edaffa030187 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:37.823579 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b10fa-421b-4493-8f48-edaffa030187 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:37.824051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b10fa-421b-4493-8f48-edaffa030187 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:37.927392 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b10fa-421b-4493-8f48-edaffa030187 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:37.929844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b10fa-421b-4493-8f48-edaffa030187 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:37.973898 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc3355-b7e7-4601-bb3b-e6de8f8fb7f5 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:37.978277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:38.041295 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fec4b9-c25f-41aa-8470-14d5c2aea9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:38.042357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fec4b9-c25f-41aa-8470-14d5c2aea9c0 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:38.042844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fec4b9-c25f-41aa-8470-14d5c2aea9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:38.043460 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fec4b9-c25f-41aa-8470-14d5c2aea9c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:38.057241 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fec4b9-c25f-41aa-8470-14d5c2aea9c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:38.060025 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fec4b9-c25f-41aa-8470-14d5c2aea9c0 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:38.063699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:38.065438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 328/662] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:09:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:09:38.066773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b10fa-421b-4493-8f48-edaffa030187 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:38.085037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-64fec4b9-c25f-41aa-8470-14d5c2aea9c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:38.085037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:38.085037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:38.085037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:38.085037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:38.085037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:38.085037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:38.085037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:38.085037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:38.085037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:38.085037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:38.085037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:38.085037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:38.085037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:38.085037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:38.086196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:38.086196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:38.086196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:38.086196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:38.086196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:38.086196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:38.086196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:38.086196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:38.086196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:38.086196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:38.086196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:38.086196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:38.086196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:38.086196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:38.086196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:38.086196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:38.086196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:38.088033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:38.088033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:38.088033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:38.088033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:38.088033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:38.088033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:38.088033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:38.088033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:38.088033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 335/663] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:38.088590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fec4b9-c25f-41aa-8470-14d5c2aea9c0 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:38.234121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b10fa-421b-4493-8f48-edaffa030187 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:38.252467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fec4b9-c25f-41aa-8470-14d5c2aea9c0 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:38.299928 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac174e9-4df4-4371-827f-f05b34414c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:38.300589 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac174e9-4df4-4371-827f-f05b34414c2d None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:38.300889 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac174e9-4df4-4371-827f-f05b34414c2d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:38.301280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac174e9-4df4-4371-827f-f05b34414c2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:38.305149 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5ecd1-de5f-4b20-a9a6-5b937e0929ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:38.306219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5ecd1-de5f-4b20-a9a6-5b937e0929ac None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:38.306671 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5ecd1-de5f-4b20-a9a6-5b937e0929ac None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:38.307160 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5ecd1-de5f-4b20-a9a6-5b937e0929ac None admin] PATH_INFO: `/v3/users/neutron` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:38.312214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:38.316235 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac174e9-4df4-4371-827f-f05b34414c2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:38.317858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac174e9-4df4-4371-827f-f05b34414c2d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:38.323103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5ecd1-de5f-4b20-a9a6-5b937e0929ac None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:38.325930 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5ecd1-de5f-4b20-a9a6-5b937e0929ac None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:38.335803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-8ac174e9-4df4-4371-827f-f05b34414c2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:38.335803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:38.335803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:38.335803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:38.335803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:38.335803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:38.335803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:38.335803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:38.335803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:38.335803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:38.335803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:38.335803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:38.335803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:38.335803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:38.335803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:38.335803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:38.336821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:38.336821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:38.336821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:38.336821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:38.336821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:38.336821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:38.336821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:38.336821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:38.336821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:38.336821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:38.336821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:38.336821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:38.336821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:38.336821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:38.336821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:38.336821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:38.336821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:38.338024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:38.338024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:38.338024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:38.344539 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 336/664] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:38.345922 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac174e9-4df4-4371-827f-f05b34414c2d None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:38.349806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-43b5ecd1-de5f-4b20-a9a6-5b937e0929ac None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 13:09:38.349806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:38.349806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:38.349806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:38.349806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:38.349806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:38.349806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:38.349806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:38.349806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:38.349806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:38.349806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:38.349806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:38.349806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:38.349806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:38.349806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:38.349806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:38.350990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:38.352247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:38.352247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:38.352247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:38.352247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:38.352247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:38.352247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:38.352247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:38.352247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:38.352247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 13:09:38.352247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:38.359367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 329/665] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:38.361438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5ecd1-de5f-4b20-a9a6-5b937e0929ac None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:38.418013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:38.502116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5ecd1-de5f-4b20-a9a6-5b937e0929ac None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:38.512250 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac174e9-4df4-4371-827f-f05b34414c2d None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:38.559346 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cbb30-9f84-455c-8143-b19c4bda8b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:38.559913 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cbb30-9f84-455c-8143-b19c4bda8b29 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:38.560200 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cbb30-9f84-455c-8143-b19c4bda8b29 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:38.560490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cbb30-9f84-455c-8143-b19c4bda8b29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:38.574555 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cbb30-9f84-455c-8143-b19c4bda8b29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:38.574873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4094b5-81a4-496f-b4b1-1ceeff312366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:38.575842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4094b5-81a4-496f-b4b1-1ceeff312366 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:38.576482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4094b5-81a4-496f-b4b1-1ceeff312366 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:38.576811 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4094b5-81a4-496f-b4b1-1ceeff312366 None admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:38.577960 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4094b5-81a4-496f-b4b1-1ceeff312366 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:38.578541 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4094b5-81a4-496f-b4b1-1ceeff312366 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:38.581349 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4094b5-81a4-496f-b4b1-1ceeff312366 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:38.582540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-033cbb30-9f84-455c-8143-b19c4bda8b29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:38.582540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:38.582540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:38.582540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:38.582540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:38.582540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:38.582540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:38.582540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:38.582540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:38.582540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:38.582540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:38.582540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:38.582540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:38.582540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:38.582540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:38.582540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:38.584598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:38.584598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:38.584598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:38.584598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:38.584598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:38.584598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:38.584598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:38.584598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:38.584598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:38.584598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:38.584598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:38.584598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:38.584598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:38.584598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:38.584598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:38.584598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:38.586255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:38.586255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:38.586255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:38.586255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:38.586255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:38.586255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:38.586255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:38.586255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:38.586255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:38.586255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:38.586255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:38.586255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:38.586255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 337/666] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:38.586255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cbb30-9f84-455c-8143-b19c4bda8b29 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:38.614541 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 330/667] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:38.616306 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4094b5-81a4-496f-b4b1-1ceeff312366 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:38.711880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:38.728137 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cbb30-9f84-455c-8143-b19c4bda8b29 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:38.744701 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:38.777244 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b539109-039f-41b7-9624-7b5d5211aba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:38.777858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b539109-039f-41b7-9624-7b5d5211aba2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:38.778139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b539109-039f-41b7-9624-7b5d5211aba2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:38.778418 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b539109-039f-41b7-9624-7b5d5211aba2 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:38.779131 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b539109-039f-41b7-9624-7b5d5211aba2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:38.779440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b539109-039f-41b7-9624-7b5d5211aba2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:38.780998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b539109-039f-41b7-9624-7b5d5211aba2 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:38.784066 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4094b5-81a4-496f-b4b1-1ceeff312366 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:38.803478 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 338/668] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:38.806040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b539109-039f-41b7-9624-7b5d5211aba2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:38.834523 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b917372-25ad-4376-ac62-575902f60bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:38.835326 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b917372-25ad-4376-ac62-575902f60bca None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:38.835756 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b917372-25ad-4376-ac62-575902f60bca None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:38.836241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b917372-25ad-4376-ac62-575902f60bca None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:38.837500 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b917372-25ad-4376-ac62-575902f60bca None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:38.837951 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b917372-25ad-4376-ac62-575902f60bca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:38.839848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b917372-25ad-4376-ac62-575902f60bca None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:38.867511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 331/669] 10.222.0.62 () {56 vars in 1139 bytes} [Wed Sep 7 13:09:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:38.870329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b917372-25ad-4376-ac62-575902f60bca None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:38.966024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b539109-039f-41b7-9624-7b5d5211aba2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:39.013797 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cb7dc-6686-4448-8a1b-45e3a8bd59fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:39.014548 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cb7dc-6686-4448-8a1b-45e3a8bd59fc None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:39.014829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cb7dc-6686-4448-8a1b-45e3a8bd59fc None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:39.015224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0cb7dc-6686-4448-8a1b-45e3a8bd59fc None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:39.017700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0cb7dc-6686-4448-8a1b-45e3a8bd59fc None admin] RBAC: Adding query filter params (name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:39.017700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0cb7dc-6686-4448-8a1b-45e3a8bd59fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:39.018678 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0cb7dc-6686-4448-8a1b-45e3a8bd59fc None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:39.022056 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b917372-25ad-4376-ac62-575902f60bca None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:39.036205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 339/670] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:09:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:39.036205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cb7dc-6686-4448-8a1b-45e3a8bd59fc None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:39.048096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:39.072569 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d6aca-2ca6-4bc3-bb15-8113c77e1ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:39.073277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d6aca-2ca6-4bc3-bb15-8113c77e1ff8 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:39.073615 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d6aca-2ca6-4bc3-bb15-8113c77e1ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:39.073895 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d6aca-2ca6-4bc3-bb15-8113c77e1ff8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:39.074807 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d6aca-2ca6-4bc3-bb15-8113c77e1ff8 None admin] RBAC: Adding query filter params (role.id=743ed2d8e7114975bd9245742a41fe0f, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=fc2fa0fa46b842ceb1cac437166b8aea) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:39.075063 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d6aca-2ca6-4bc3-bb15-8113c77e1ff8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:39.076577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d6aca-2ca6-4bc3-bb15-8113c77e1ff8 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:39.086098 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 332/671] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:09:38 2022] GET /identity/v3/role_assignments?user.id=fc2fa0fa46b842ceb1cac437166b8aea&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=743ed2d8e7114975bd9245742a41fe0f => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:39.086724 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d6aca-2ca6-4bc3-bb15-8113c77e1ff8 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:39.103400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:39.212305 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0cb7dc-6686-4448-8a1b-45e3a8bd59fc None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:39.225169 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d6aca-2ca6-4bc3-bb15-8113c77e1ff8 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:39.271842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee15cb5-4982-4c2e-8ae4-de6d52b79a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:39.272830 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee15cb5-4982-4c2e-8ae4-de6d52b79a19 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:39.273183 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee15cb5-4982-4c2e-8ae4-de6d52b79a19 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:39.273633 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee15cb5-4982-4c2e-8ae4-de6d52b79a19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:39.278381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504fe749-62a2-42ea-a042-7861a3b78cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:39.279792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fe749-62a2-42ea-a042-7861a3b78cac None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:39.280175 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fe749-62a2-42ea-a042-7861a3b78cac None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:39.280513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fe749-62a2-42ea-a042-7861a3b78cac None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:39.293019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504fe749-62a2-42ea-a042-7861a3b78cac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:39.295626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504fe749-62a2-42ea-a042-7861a3b78cac None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:39.318363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-504fe749-62a2-42ea-a042-7861a3b78cac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:39.318363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:39.318363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:39.318363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:39.318363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:39.318363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:39.318363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:39.318363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:39.318363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:39.318363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:39.318363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:39.318363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:39.318363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:39.318363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:39.318363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:39.319702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:39.319702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:39.319702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:39.319702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:39.319702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:39.319702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:39.319702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:39.319702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:39.319702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:39.319702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:39.319702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:39.319702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:39.319702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:39.319702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:39.319702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:39.319702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:39.319702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:39.320628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:39.320628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:39.320628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:39.320628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:39.320628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:39.320628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:39.320628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:39.320628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:39.323432 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 340/672] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:39.324912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504fe749-62a2-42ea-a042-7861a3b78cac None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:39.369413 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:39.419667 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee15cb5-4982-4c2e-8ae4-de6d52b79a19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:39.421808 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee15cb5-4982-4c2e-8ae4-de6d52b79a19 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:39.459666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:39.476322 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504fe749-62a2-42ea-a042-7861a3b78cac None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:39.543534 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd595ab7-f14e-46a7-a1f5-2143cbe5ea62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:39.544492 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd595ab7-f14e-46a7-a1f5-2143cbe5ea62 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:39.545036 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd595ab7-f14e-46a7-a1f5-2143cbe5ea62 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:39.545463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd595ab7-f14e-46a7-a1f5-2143cbe5ea62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:39.562256 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd595ab7-f14e-46a7-a1f5-2143cbe5ea62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:39.565936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd595ab7-f14e-46a7-a1f5-2143cbe5ea62 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:39.599587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-dd595ab7-f14e-46a7-a1f5-2143cbe5ea62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:39.599587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:39.599587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:39.599587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:39.599587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:39.599587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:39.599587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:39.599587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:39.599587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:39.599587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:39.599587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:39.599587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:39.599587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:39.599587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:39.599587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:39.599587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:39.600904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:39.600904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:39.600904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:39.600904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:39.600904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:39.600904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:39.600904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:39.600904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:39.600904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:39.600904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:39.600904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:39.600904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:39.600904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:39.600904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:39.600904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:39.600904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:39.600904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:39.602287 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:39.602287 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:39.602287 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:39.604352 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 341/673] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:39.605909 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd595ab7-f14e-46a7-a1f5-2143cbe5ea62 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:39.649438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 333/674] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:09:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:09:39.651022 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee15cb5-4982-4c2e-8ae4-de6d52b79a19 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:39.744704 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd595ab7-f14e-46a7-a1f5-2143cbe5ea62 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:39.798231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:39.808989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d495d8f-4573-40ef-a4ed-d43666ab5435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:39.809768 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d495d8f-4573-40ef-a4ed-d43666ab5435 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:39.810148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d495d8f-4573-40ef-a4ed-d43666ab5435 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:39.810543 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d495d8f-4573-40ef-a4ed-d43666ab5435 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:39.812063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d495d8f-4573-40ef-a4ed-d43666ab5435 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:39.812523 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d495d8f-4573-40ef-a4ed-d43666ab5435 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:39.815617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d495d8f-4573-40ef-a4ed-d43666ab5435 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:39.835186 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 342/675] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:39.836236 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d495d8f-4573-40ef-a4ed-d43666ab5435 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:39.837412 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee15cb5-4982-4c2e-8ae4-de6d52b79a19 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:39.841749 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:39.919029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f833262-c9be-49fa-ba2b-4a834cbf905c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:39.919835 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f833262-c9be-49fa-ba2b-4a834cbf905c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:39.920229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f833262-c9be-49fa-ba2b-4a834cbf905c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:39.920621 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f833262-c9be-49fa-ba2b-4a834cbf905c None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:39.921816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f833262-c9be-49fa-ba2b-4a834cbf905c None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:39.922812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f833262-c9be-49fa-ba2b-4a834cbf905c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:39.924486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f833262-c9be-49fa-ba2b-4a834cbf905c None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:39.942844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 334/676] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:39.944518 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f833262-c9be-49fa-ba2b-4a834cbf905c None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:39.999917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d495d8f-4573-40ef-a4ed-d43666ab5435 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:40.053505 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5024e-2df7-418a-ac99-6934b5094d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:40.054935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5024e-2df7-418a-ac99-6934b5094d0e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:40.055673 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5024e-2df7-418a-ac99-6934b5094d0e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:40.056360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5024e-2df7-418a-ac99-6934b5094d0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:40.077667 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5024e-2df7-418a-ac99-6934b5094d0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:40.079628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5024e-2df7-418a-ac99-6934b5094d0e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:40.102174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-2ed5024e-2df7-418a-ac99-6934b5094d0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:40.102174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:40.102174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:40.102174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:40.102174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:40.102174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:40.102174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:40.102174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:40.102174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:40.102174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:40.102174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:40.102174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:40.102174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:40.102174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:40.102174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:40.102174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:40.104391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:40.104391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:40.104391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:40.104391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:40.104391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:40.104391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:40.104391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:40.104391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:40.104391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:40.104391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:40.104391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:40.104391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:40.104391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:40.104391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:40.104391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:40.104391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:40.104391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:40.106815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:40.106815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:40.106815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:40.106815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 343/677] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:40.107441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5024e-2df7-418a-ac99-6934b5094d0e None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:40.135458 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f833262-c9be-49fa-ba2b-4a834cbf905c None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:40.193014 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61bc44-0d07-4c7f-aa1a-0eb4d95455f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:40.193540 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61bc44-0d07-4c7f-aa1a-0eb4d95455f5 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:40.193918 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61bc44-0d07-4c7f-aa1a-0eb4d95455f5 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:40.194193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61bc44-0d07-4c7f-aa1a-0eb4d95455f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:40.195042 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61bc44-0d07-4c7f-aa1a-0eb4d95455f5 None admin] RBAC: Adding query filter params (role.id=743ed2d8e7114975bd9245742a41fe0f, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=97c363046db444cbb3d4346066f9ad42) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:40.195357 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61bc44-0d07-4c7f-aa1a-0eb4d95455f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:40.197841 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61bc44-0d07-4c7f-aa1a-0eb4d95455f5 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:40.210923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 335/678] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:09:39 2022] GET /identity/v3/role_assignments?user.id=97c363046db444cbb3d4346066f9ad42&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=743ed2d8e7114975bd9245742a41fe0f => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:40.212546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61bc44-0d07-4c7f-aa1a-0eb4d95455f5 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:40.217176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:40.219175 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:40.295126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5024e-2df7-418a-ac99-6934b5094d0e None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:40.355599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da6c81-c25a-4c1b-bbf1-774fea14a021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:40.357060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6c81-c25a-4c1b-bbf1-774fea14a021 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:40.357744 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6c81-c25a-4c1b-bbf1-774fea14a021 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:40.358452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6c81-c25a-4c1b-bbf1-774fea14a021 None admin] PATH_INFO: `/v3/users/swift` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:40.376293 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da6c81-c25a-4c1b-bbf1-774fea14a021 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:40.379495 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da6c81-c25a-4c1b-bbf1-774fea14a021 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:40.397291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-f9da6c81-c25a-4c1b-bbf1-774fea14a021 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:09:40.397291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:40.397291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:40.397291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:40.397291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:40.397291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:40.397291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:40.397291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:40.397291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:40.397291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:40.397291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:40.397291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:40.397291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:40.397291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:40.397291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:40.397291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:40.398775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:40.400216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:40.400216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:40.400216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:40.400216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:40.400216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:40.400216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:40.400216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:40.400216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:40.400216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:09:40.400216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:40.403041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 344/679] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:09:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:40.405010 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ab738-4ee1-4ec5-b41f-70580cf56306 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:40.405564 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ab738-4ee1-4ec5-b41f-70580cf56306 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:40.407260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ab738-4ee1-4ec5-b41f-70580cf56306 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:40.409639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 345/680] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:40.411167 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ab738-4ee1-4ec5-b41f-70580cf56306 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:40.454887 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61bc44-0d07-4c7f-aa1a-0eb4d95455f5 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:40.500715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125e130-682b-476a-bec5-a21ea654f6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:40.501364 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125e130-682b-476a-bec5-a21ea654f6b5 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:40.501670 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125e130-682b-476a-bec5-a21ea654f6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:40.502027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125e130-682b-476a-bec5-a21ea654f6b5 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:40.502606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1125e130-682b-476a-bec5-a21ea654f6b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:40.502912 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1125e130-682b-476a-bec5-a21ea654f6b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:40.504953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1125e130-682b-476a-bec5-a21ea654f6b5 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:40.526179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 336/681] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:40.527228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49596cf-3a1c-42e2-8b00-e9a6c970c085 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:40.527519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49596cf-3a1c-42e2-8b00-e9a6c970c085 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:40.527791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49596cf-3a1c-42e2-8b00-e9a6c970c085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:40.568581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ab738-4ee1-4ec5-b41f-70580cf56306 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:40.576697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-b49596cf-3a1c-42e2-8b00-e9a6c970c085 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:40.579074 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b49596cf-3a1c-42e2-8b00-e9a6c970c085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:40.617373 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:40.650220 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96063c22-67f7-483f-91dd-28ae8d3d3aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:40.652170 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96063c22-67f7-483f-91dd-28ae8d3d3aef None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:40.652170 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96063c22-67f7-483f-91dd-28ae8d3d3aef None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:40.652170 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96063c22-67f7-483f-91dd-28ae8d3d3aef None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:40.653449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96063c22-67f7-483f-91dd-28ae8d3d3aef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:40.653937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96063c22-67f7-483f-91dd-28ae8d3d3aef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:40.657185 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96063c22-67f7-483f-91dd-28ae8d3d3aef None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:40.702390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 346/682] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Sep 7 13:09:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:40.702998 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:40.705060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96063c22-67f7-483f-91dd-28ae8d3d3aef None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:40.722591 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 337/683] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:40.724205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49596cf-3a1c-42e2-8b00-e9a6c970c085 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:40.961209 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49596cf-3a1c-42e2-8b00-e9a6c970c085 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:41.030156 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96063c22-67f7-483f-91dd-28ae8d3d3aef None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:41.049931 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676ce51-3071-4b19-8456-b545332b09c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:41.050564 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676ce51-3071-4b19-8456-b545332b09c4 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:41.050843 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676ce51-3071-4b19-8456-b545332b09c4 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:41.051149 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676ce51-3071-4b19-8456-b545332b09c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:41.111301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d9a1c-7bf3-4f1c-a7b8-1b1774b8ce62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:41.112343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9a1c-7bf3-4f1c-a7b8-1b1774b8ce62 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:41.112798 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9a1c-7bf3-4f1c-a7b8-1b1774b8ce62 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:41.113267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9a1c-7bf3-4f1c-a7b8-1b1774b8ce62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:41.136145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d9a1c-7bf3-4f1c-a7b8-1b1774b8ce62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:41.141905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d9a1c-7bf3-4f1c-a7b8-1b1774b8ce62 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:41.182544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-b95d9a1c-7bf3-4f1c-a7b8-1b1774b8ce62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:41.182544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:41.182544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:41.182544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:41.182544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:41.182544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:41.182544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:41.182544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:41.182544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:41.182544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:41.182544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:41.182544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:41.182544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:41.182544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:41.182544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:41.184270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:41.184270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:41.184270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:41.184270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:41.184270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:41.184270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:41.184270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:41.184270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:41.184270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:41.184270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:41.184270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:41.184270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:41.184270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:41.184270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:41.184270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:41.184270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:41.184270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:41.186535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:41.186535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:41.186535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:41.186535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:41.186535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:41.186535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:41.186535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:41.186535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:41.192627 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 347/684] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 485 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:41.197837 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d9a1c-7bf3-4f1c-a7b8-1b1774b8ce62 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:41.231170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:41.259281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676ce51-3071-4b19-8456-b545332b09c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:41.261764 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676ce51-3071-4b19-8456-b545332b09c4 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:41.289662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:41.413812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d9a1c-7bf3-4f1c-a7b8-1b1774b8ce62 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:41.478879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ec418-afbc-4ff0-bb6a-25e8d2bade20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:41.478879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ec418-afbc-4ff0-bb6a-25e8d2bade20 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:41.478879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ec418-afbc-4ff0-bb6a-25e8d2bade20 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:41.478879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ec418-afbc-4ff0-bb6a-25e8d2bade20 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:41.483649 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ec418-afbc-4ff0-bb6a-25e8d2bade20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:41.484358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ec418-afbc-4ff0-bb6a-25e8d2bade20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:41.487245 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ec418-afbc-4ff0-bb6a-25e8d2bade20 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:41.492480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 338/685] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:09:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 769 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:09:41.493383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676ce51-3071-4b19-8456-b545332b09c4 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:41.513439 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 348/686] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:09:41.515421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ec418-afbc-4ff0-bb6a-25e8d2bade20 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:41.689106 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:41.704835 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676ce51-3071-4b19-8456-b545332b09c4 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:41.717560 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:41.732524 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ec418-afbc-4ff0-bb6a-25e8d2bade20 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:41.776140 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943a8af-d043-4079-986d-edcf18831e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:41.777955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943a8af-d043-4079-986d-edcf18831e66 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:41.778628 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943a8af-d043-4079-986d-edcf18831e66 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:41.779324 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943a8af-d043-4079-986d-edcf18831e66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:41.802670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef17f23-0814-4ff1-a108-bef9b67ad313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:41.803782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef17f23-0814-4ff1-a108-bef9b67ad313 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:41.804241 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef17f23-0814-4ff1-a108-bef9b67ad313 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:41.804683 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef17f23-0814-4ff1-a108-bef9b67ad313 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:41.804849 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943a8af-d043-4079-986d-edcf18831e66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:41.807909 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943a8af-d043-4079-986d-edcf18831e66 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:41.826024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef17f23-0814-4ff1-a108-bef9b67ad313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:41.830465 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef17f23-0814-4ff1-a108-bef9b67ad313 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:41.841605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-d943a8af-d043-4079-986d-edcf18831e66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:41.841605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:41.841605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:41.841605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:41.841605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:41.841605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:41.841605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:41.841605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:41.841605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:41.841605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:41.841605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:41.841605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:41.841605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:41.841605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:41.841605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:41.841605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:41.843715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:41.843715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:41.843715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:41.843715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:41.843715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:41.843715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:41.843715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:41.843715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:41.843715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:41.843715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:41.843715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:41.843715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:41.843715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:41.843715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:41.843715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:41.843715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:41.843715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:41.846272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:41.846272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:41.846272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:41.858716 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 339/687] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:41.858716 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943a8af-d043-4079-986d-edcf18831e66 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:41.873331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-3ef17f23-0814-4ff1-a108-bef9b67ad313 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:41.873331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:41.873331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:41.873331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:41.873331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:41.873331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:41.873331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:41.873331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:41.873331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:41.873331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:41.873331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:41.873331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:41.873331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:41.873331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:41.873331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:41.873331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:41.876126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:41.876126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:41.876126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:41.876126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:41.876126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:41.876126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:41.876126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:41.876126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:41.876126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:41.876126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:41.876126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:41.876126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:41.876126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:41.876126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:41.876126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:41.876126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:41.876126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:41.877645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:41.877645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:41.877645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:41.886494 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 349/688] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:41.888131 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef17f23-0814-4ff1-a108-bef9b67ad313 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:42.082154 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef17f23-0814-4ff1-a108-bef9b67ad313 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:42.086893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943a8af-d043-4079-986d-edcf18831e66 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:42.152481 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dc755-5a22-4e52-8dc3-b6c413b77780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:42.153578 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc755-5a22-4e52-8dc3-b6c413b77780 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:42.154102 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc755-5a22-4e52-8dc3-b6c413b77780 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:42.154716 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc755-5a22-4e52-8dc3-b6c413b77780 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:42.157252 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dc755-5a22-4e52-8dc3-b6c413b77780 None admin] RBAC: Adding query filter params (role.id=743ed2d8e7114975bd9245742a41fe0f, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=5acdbdb6f23b4f10bcbab0387ea67ddf) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:42.157677 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dc755-5a22-4e52-8dc3-b6c413b77780 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:42.160600 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dc755-5a22-4e52-8dc3-b6c413b77780 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:42.163101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa073a53-388f-4c44-b4d2-67cd77ff85cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:42.164911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa073a53-388f-4c44-b4d2-67cd77ff85cf None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:42.165649 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa073a53-388f-4c44-b4d2-67cd77ff85cf None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:42.167107 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa073a53-388f-4c44-b4d2-67cd77ff85cf None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:42.168381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa073a53-388f-4c44-b4d2-67cd77ff85cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:42.168898 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa073a53-388f-4c44-b4d2-67cd77ff85cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:42.172016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa073a53-388f-4c44-b4d2-67cd77ff85cf None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:42.179030 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 340/689] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:09:41 2022] GET /identity/v3/role_assignments?user.id=5acdbdb6f23b4f10bcbab0387ea67ddf&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=743ed2d8e7114975bd9245742a41fe0f => generated 608 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:42.179030 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dc755-5a22-4e52-8dc3-b6c413b77780 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:42.205756 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:42.215034 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 350/690] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:42.221829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa073a53-388f-4c44-b4d2-67cd77ff85cf None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:42.252056 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:42.419516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa073a53-388f-4c44-b4d2-67cd77ff85cf None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:42.423661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dc755-5a22-4e52-8dc3-b6c413b77780 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:42.467991 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b7fe9-09db-4bf6-81c0-04a8f7bc6902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:42.470490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b7fe9-09db-4bf6-81c0-04a8f7bc6902 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:42.470823 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b7fe9-09db-4bf6-81c0-04a8f7bc6902 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:42.471344 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b7fe9-09db-4bf6-81c0-04a8f7bc6902 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:42.472719 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b7fe9-09db-4bf6-81c0-04a8f7bc6902 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:42.479385 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b7fe9-09db-4bf6-81c0-04a8f7bc6902 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:42.503154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2c4ed-d062-4cb5-b516-b7f363a29f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:42.505130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c4ed-d062-4cb5-b516-b7f363a29f29 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:42.505130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c4ed-d062-4cb5-b516-b7f363a29f29 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:42.505130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c4ed-d062-4cb5-b516-b7f363a29f29 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:42.505782 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2c4ed-d062-4cb5-b516-b7f363a29f29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:42.506231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2c4ed-d062-4cb5-b516-b7f363a29f29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:42.508546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2c4ed-d062-4cb5-b516-b7f363a29f29 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:42.528687 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 341/691] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:42.530915 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0f04c-b100-4da5-b133-bcee966a0f75 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:42.536977 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0f04c-b100-4da5-b133-bcee966a0f75 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:42.536977 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0f04c-b100-4da5-b133-bcee966a0f75 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:42.536977 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 342/692] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:42.541227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0f04c-b100-4da5-b133-bcee966a0f75 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:42.593577 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 351/693] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:09:42 2022] POST /identity/v3/users => generated 254 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:09:42.596372 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35fdfa-ec64-4066-84fa-3c6f766ae6aa None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:42.596372 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35fdfa-ec64-4066-84fa-3c6f766ae6aa None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:42.596839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35fdfa-ec64-4066-84fa-3c6f766ae6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:42.621450 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-7a35fdfa-ec64-4066-84fa-3c6f766ae6aa None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:42.622706 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7a35fdfa-ec64-4066-84fa-3c6f766ae6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:42.683490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:42.781257 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0f04c-b100-4da5-b133-bcee966a0f75 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:42.825040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:42.851558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 352/694] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:42.866879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8ce74-c1c4-4cb8-93fb-da18cae59494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:42.867672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8ce74-c1c4-4cb8-93fb-da18cae59494 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:42.867980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8ce74-c1c4-4cb8-93fb-da18cae59494 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:42.868268 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8ce74-c1c4-4cb8-93fb-da18cae59494 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:42.879815 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8ce74-c1c4-4cb8-93fb-da18cae59494 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:42.882194 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8ce74-c1c4-4cb8-93fb-da18cae59494 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:42.900561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-3bc8ce74-c1c4-4cb8-93fb-da18cae59494 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:42.900561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:42.900561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:42.900561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:42.900561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:42.900561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:42.900561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:42.900561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:42.900561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:42.900561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:42.900561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:42.900561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:42.900561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:42.900561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:42.900561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:42.901548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:42.901548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:42.901548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:42.901548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:42.901548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:42.901548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:42.901548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:42.901548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:42.901548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:42.901548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:42.901548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:42.901548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:42.901548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:42.901548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:42.901548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:42.901548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:42.901548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:42.902434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:42.902434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:42.902434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:42.902434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:42.902434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:42.902434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:42.902434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:42.902434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:42.902434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 343/695] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:42.911270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35fdfa-ec64-4066-84fa-3c6f766ae6aa None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:43.068258 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35fdfa-ec64-4066-84fa-3c6f766ae6aa None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:43.129096 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061be32-b652-482e-a7f7-0af49364f4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:43.130068 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061be32-b652-482e-a7f7-0af49364f4f3 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:43.130490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061be32-b652-482e-a7f7-0af49364f4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:43.130908 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061be32-b652-482e-a7f7-0af49364f4f3 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:43.132087 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061be32-b652-482e-a7f7-0af49364f4f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:43.132618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061be32-b652-482e-a7f7-0af49364f4f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:43.135033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061be32-b652-482e-a7f7-0af49364f4f3 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:43.158168 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 353/696] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:43.167976 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8ce74-c1c4-4cb8-93fb-da18cae59494 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:43.300422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061be32-b652-482e-a7f7-0af49364f4f3 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:43.403254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8ce74-c1c4-4cb8-93fb-da18cae59494 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:43.418223 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:43.471566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:43.488172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e3beb-472e-4e44-9cc0-f65858a5874c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:43.489919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e3beb-472e-4e44-9cc0-f65858a5874c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:43.490686 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e3beb-472e-4e44-9cc0-f65858a5874c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:43.492612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e3beb-472e-4e44-9cc0-f65858a5874c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:43.494453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e3beb-472e-4e44-9cc0-f65858a5874c None admin] RBAC: Adding query filter params (role.id=743ed2d8e7114975bd9245742a41fe0f, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=5cedc7091d2f46b5be68818d26399e9b) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:43.495897 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e3beb-472e-4e44-9cc0-f65858a5874c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:43.500991 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e3beb-472e-4e44-9cc0-f65858a5874c None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:43.511158 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 344/697] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:09:43 2022] GET /identity/v3/role_assignments?user.id=5cedc7091d2f46b5be68818d26399e9b&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=743ed2d8e7114975bd9245742a41fe0f => generated 608 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:43.513506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b4752-5db1-45d4-9907-faa80f23d6f9 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:43.513992 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b4752-5db1-45d4-9907-faa80f23d6f9 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:43.514444 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b4752-5db1-45d4-9907-faa80f23d6f9 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:43.530454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061be32-b652-482e-a7f7-0af49364f4f3 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:43.535077 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 345/698] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:43.541670 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c3824-fe71-46b1-b28b-f9bc7e5896bc None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:43.542123 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c3824-fe71-46b1-b28b-f9bc7e5896bc None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:43.542589 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c3824-fe71-46b1-b28b-f9bc7e5896bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:43.584292 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-5a3c3824-fe71-46b1-b28b-f9bc7e5896bc None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:43.586241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5a3c3824-fe71-46b1-b28b-f9bc7e5896bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:43.667670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784c9df-808e-4f94-9429-1eb14748bfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:43.668649 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784c9df-808e-4f94-9429-1eb14748bfa0 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:43.669135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784c9df-808e-4f94-9429-1eb14748bfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:43.669609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784c9df-808e-4f94-9429-1eb14748bfa0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:43.670724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784c9df-808e-4f94-9429-1eb14748bfa0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:43.673377 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784c9df-808e-4f94-9429-1eb14748bfa0 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:43.723230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-1784c9df-808e-4f94-9429-1eb14748bfa0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:09:43.723230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:43.723230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:09:43.723230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:09:43.723230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:09:43.723230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:09:43.723230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:09:43.723230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:09:43.723230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:09:43.723230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:43.723230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:09:43.723230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:43.723230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:43.723230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:43.723230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:43.723230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:43.723230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:43.725207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:43.725207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:43.725207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:43.725207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:43.725207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:43.725207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:43.725207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:09:43.725207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:09:43.725207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:09:43.725207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:09:43.725207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:43.725207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:43.725207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:43.725207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:43.725207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:43.725207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:43.726812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:43.726812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:43.726812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:43.726812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:43.726812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:43.726812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:43.726812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:43.726812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:43.726812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:43.726812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:43.726812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:09:43.726812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:09:43.726812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:09:43.726812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:43.726812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 354/699] 10.222.0.62 () {56 vars in 1129 bytes} [Wed Sep 7 13:09:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:43.747053 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784c9df-808e-4f94-9429-1eb14748bfa0 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:43.795332 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 346/700] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:43.946933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784c9df-808e-4f94-9429-1eb14748bfa0 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:43.979990 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:44.012360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247664f0-180e-4e3c-b625-bc8afdda4442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:44.013181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247664f0-180e-4e3c-b625-bc8afdda4442 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:44.013459 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247664f0-180e-4e3c-b625-bc8afdda4442 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:44.013731 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247664f0-180e-4e3c-b625-bc8afdda4442 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:44.014505 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247664f0-180e-4e3c-b625-bc8afdda4442 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:44.014810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247664f0-180e-4e3c-b625-bc8afdda4442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:44.017313 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247664f0-180e-4e3c-b625-bc8afdda4442 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:44.024295 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 355/701] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Sep 7 13:09:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:44.029347 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c3824-fe71-46b1-b28b-f9bc7e5896bc None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:44.122652 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247664f0-180e-4e3c-b625-bc8afdda4442 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:44.214923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c3824-fe71-46b1-b28b-f9bc7e5896bc None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:44.271020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:44.301450 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5378d37-a1e4-4398-978b-3780ad70f6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:44.302190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5378d37-a1e4-4398-978b-3780ad70f6f3 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:44.302514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5378d37-a1e4-4398-978b-3780ad70f6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:44.302816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5378d37-a1e4-4398-978b-3780ad70f6f3 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:44.303784 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5378d37-a1e4-4398-978b-3780ad70f6f3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:44.304420 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5378d37-a1e4-4398-978b-3780ad70f6f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:44.305926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5378d37-a1e4-4398-978b-3780ad70f6f3 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:44.320731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 347/702] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Sep 7 13:09:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:44.335324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247664f0-180e-4e3c-b625-bc8afdda4442 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:44.408555 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c7209-b152-478b-90bf-20e447828696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:44.409431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c7209-b152-478b-90bf-20e447828696 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:44.409716 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c7209-b152-478b-90bf-20e447828696 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:44.410000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c7209-b152-478b-90bf-20e447828696 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:44.428402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c7209-b152-478b-90bf-20e447828696 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:44.433536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-4d3c7209-b152-478b-90bf-20e447828696 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:09:44.433536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:44.433536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:44.433536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:44.433536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:44.433536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:44.433536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:44.433536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:44.433536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:44.433536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:44.433536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:44.433536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:44.433536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:44.433536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:44.433536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:44.433536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:44.434834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:44.434834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:44.434834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:44.434834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:44.434834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:44.434834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:44.434834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:44.434834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:44.434834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:44.434834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:44.434834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:44.434834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:44.434834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:44.434834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:44.434834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:44.434834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:44.437160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:44.437160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:44.437160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:44.437160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:44.437160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:44.437160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:44.437160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:44.437160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:44.437160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:44.437160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:44.437160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:09:44.437160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:44.437160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 356/703] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:09:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:44.444195 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5378d37-a1e4-4398-978b-3780ad70f6f3 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:44.633652 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5378d37-a1e4-4398-978b-3780ad70f6f3 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:44.678679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d871fa-6adf-4de6-81cf-979afd478c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:44.679517 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d871fa-6adf-4de6-81cf-979afd478c00 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:44.679812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d871fa-6adf-4de6-81cf-979afd478c00 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:44.680094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d871fa-6adf-4de6-81cf-979afd478c00 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:44.680779 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d871fa-6adf-4de6-81cf-979afd478c00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:44.681082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d871fa-6adf-4de6-81cf-979afd478c00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:44.682677 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d871fa-6adf-4de6-81cf-979afd478c00 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:44.697421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 348/704] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:09:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:44.708366 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c7209-b152-478b-90bf-20e447828696 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:44.786115 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:44.881613 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c7209-b152-478b-90bf-20e447828696 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:44.946120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f1bbc-fbe4-42ff-9902-5dec7efc5102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:44.946821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f1bbc-fbe4-42ff-9902-5dec7efc5102 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:44.947192 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f1bbc-fbe4-42ff-9902-5dec7efc5102 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:44.947465 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f1bbc-fbe4-42ff-9902-5dec7efc5102 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:44.965456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f1bbc-fbe4-42ff-9902-5dec7efc5102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:44.966927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f1bbc-fbe4-42ff-9902-5dec7efc5102 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:44.999586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-880f1bbc-fbe4-42ff-9902-5dec7efc5102 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:44.999586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:44.999586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:44.999586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:44.999586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:44.999586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:44.999586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:44.999586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:44.999586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:44.999586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:44.999586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:44.999586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:44.999586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:44.999586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:44.999586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:44.999586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:45.001390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:45.001390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:45.001390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:45.001390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:45.001390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:45.001390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:45.001390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:45.001390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:45.001390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:45.001390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:45.001390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:45.001390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:45.001390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:45.001390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:45.001390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:45.001390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:45.001390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:45.004240 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:45.004240 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:45.004240 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:45.004240 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 357/705] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:09:45.008220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d871fa-6adf-4de6-81cf-979afd478c00 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:45.095564 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:45.182534 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d871fa-6adf-4de6-81cf-979afd478c00 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:45.234750 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab6810-9a3f-4a03-8bcf-e3bbf8bd8d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:45.235428 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab6810-9a3f-4a03-8bcf-e3bbf8bd8d7c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:45.235845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab6810-9a3f-4a03-8bcf-e3bbf8bd8d7c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:45.236129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab6810-9a3f-4a03-8bcf-e3bbf8bd8d7c None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:45.236929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab6810-9a3f-4a03-8bcf-e3bbf8bd8d7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:45.237244 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab6810-9a3f-4a03-8bcf-e3bbf8bd8d7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:45.238823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab6810-9a3f-4a03-8bcf-e3bbf8bd8d7c None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:45.259959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 349/706] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:45.267783 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f1bbc-fbe4-42ff-9902-5dec7efc5102 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:45.459746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:45.471258 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f1bbc-fbe4-42ff-9902-5dec7efc5102 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:45.532731 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b5369-909a-4043-b22c-f231643dee46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:45.533589 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b5369-909a-4043-b22c-f231643dee46 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:45.533866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b5369-909a-4043-b22c-f231643dee46 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:45.534140 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b5369-909a-4043-b22c-f231643dee46 None admin] PATH_INFO: `/v3/users/nova` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:45.543913 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b5369-909a-4043-b22c-f231643dee46 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:45.545475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b5369-909a-4043-b22c-f231643dee46 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:45.559499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-bf4b5369-909a-4043-b22c-f231643dee46 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:09:45.559499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:45.559499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:45.559499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:45.559499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:45.559499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:45.559499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:45.559499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:45.559499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:45.559499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:45.559499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:45.559499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:45.559499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:45.559499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:45.559499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:45.559499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:45.560545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:45.561509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:45.561509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:45.561509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:45.561509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:45.561509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:45.561509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:45.561509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:45.561509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:45.561509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:09:45.561509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:45.561509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 358/707] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 7 13:09:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:45.567571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab6810-9a3f-4a03-8bcf-e3bbf8bd8d7c None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:45.584178 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb28808-ba49-4797-a0a9-241449ef7fd4 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:45.584602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb28808-ba49-4797-a0a9-241449ef7fd4 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:45.585014 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb28808-ba49-4797-a0a9-241449ef7fd4 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:45.588240 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 359/708] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:45.601995 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef306d-9c8b-436f-81eb-993dedffb526 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:45.602641 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef306d-9c8b-436f-81eb-993dedffb526 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:45.603057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef306d-9c8b-436f-81eb-993dedffb526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:45.643961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-eaef306d-9c8b-436f-81eb-993dedffb526 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:45.645290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-eaef306d-9c8b-436f-81eb-993dedffb526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:45.745059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab6810-9a3f-4a03-8bcf-e3bbf8bd8d7c None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:45.758158 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:45.797819 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1803b-b16e-408e-a54a-69e70ee54736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:45.798638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1803b-b16e-408e-a54a-69e70ee54736 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:45.798922 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1803b-b16e-408e-a54a-69e70ee54736 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:45.799256 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1803b-b16e-408e-a54a-69e70ee54736 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:45.800126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1803b-b16e-408e-a54a-69e70ee54736 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:45.800657 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1803b-b16e-408e-a54a-69e70ee54736 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:45.802841 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1803b-b16e-408e-a54a-69e70ee54736 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:45.843092 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 350/709] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 7 13:09:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:45.848867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 360/710] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:45.851002 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1803b-b16e-408e-a54a-69e70ee54736 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:46.026696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1803b-b16e-408e-a54a-69e70ee54736 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:46.092578 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d439c6-06a4-489e-b160-5262305cdf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:46.093259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d439c6-06a4-489e-b160-5262305cdf93 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:46.093548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d439c6-06a4-489e-b160-5262305cdf93 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:46.094375 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d439c6-06a4-489e-b160-5262305cdf93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:46.147405 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:46.194595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d439c6-06a4-489e-b160-5262305cdf93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:46.196391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d439c6-06a4-489e-b160-5262305cdf93 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:46.284907 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7af4d4-9668-4ca8-8121-115eed2cf028 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:46.285107 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7af4d4-9668-4ca8-8121-115eed2cf028 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:46.285712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7af4d4-9668-4ca8-8121-115eed2cf028 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:46.288510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 361/711] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:46.295252 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f722d-ac12-47b3-893d-8d438d2b7db2 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:46.295879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f722d-ac12-47b3-893d-8d438d2b7db2 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:46.296878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f722d-ac12-47b3-893d-8d438d2b7db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:46.331642 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-994f722d-ac12-47b3-893d-8d438d2b7db2 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:46.333006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-994f722d-ac12-47b3-893d-8d438d2b7db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:46.377404 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 351/712] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:09:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 13:09:46.378871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d439c6-06a4-489e-b160-5262305cdf93 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:46.459679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 362/713] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:46.460884 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f722d-ac12-47b3-893d-8d438d2b7db2 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:46.483555 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:46.508523 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:46.608551 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f722d-ac12-47b3-893d-8d438d2b7db2 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:46.644399 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d439c6-06a4-489e-b160-5262305cdf93 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:46.670268 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db1e2d-9ff2-4fb0-9d99-741e99f8effd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:46.671451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db1e2d-9ff2-4fb0-9d99-741e99f8effd None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:46.672080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db1e2d-9ff2-4fb0-9d99-741e99f8effd None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:46.672622 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db1e2d-9ff2-4fb0-9d99-741e99f8effd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:46.700859 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db1e2d-9ff2-4fb0-9d99-741e99f8effd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:46.702682 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db1e2d-9ff2-4fb0-9d99-741e99f8effd None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:46.742535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-09db1e2d-9ff2-4fb0-9d99-741e99f8effd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:46.742535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:46.742535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:46.742535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:46.742535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:46.742535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:46.742535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:46.742535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:46.742535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:46.742535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:46.742535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:46.742535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:46.742535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:46.742535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:46.742535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:46.742535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:46.743580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:46.743580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:46.743580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:46.743580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:46.743580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:46.743580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:46.743580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:46.743580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:46.743580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:46.743580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:46.743580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:46.743580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:46.743580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:46.743580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:46.743580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:46.743580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:46.743580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:46.744499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:46.744499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:46.744499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:46.748268 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1be36a-c097-4c40-b3ab-5424b28d8969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:46.749627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1be36a-c097-4c40-b3ab-5424b28d8969 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:46.750249 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1be36a-c097-4c40-b3ab-5424b28d8969 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:46.750746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1be36a-c097-4c40-b3ab-5424b28d8969 None admin] PATH_INFO: `/v3/services/network` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:46.751804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1be36a-c097-4c40-b3ab-5424b28d8969 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:46.753583 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1be36a-c097-4c40-b3ab-5424b28d8969 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:46.756547 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 363/714] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:46.764081 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db1e2d-9ff2-4fb0-9d99-741e99f8effd None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:46.782896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-8a1be36a-c097-4c40-b3ab-5424b28d8969 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:09:46.782896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:46.782896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:09:46.782896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:09:46.782896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:09:46.782896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:09:46.782896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:09:46.782896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:09:46.782896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:09:46.782896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:46.782896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:09:46.782896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:46.782896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:46.782896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:46.782896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:46.782896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:46.782896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:46.784043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:46.784043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:46.784043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:46.784043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:46.784043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:46.784043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:46.784043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:09:46.784043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:09:46.784043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:09:46.784043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:09:46.784043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:46.784043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:46.784043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:46.784043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:46.784043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:46.784043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:46.784916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:46.784916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:46.784916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:46.784916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:46.784916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:46.784916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:46.784916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:46.784916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:46.784916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:46.784916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:46.784916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:09:46.784916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:09:46.784916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:09:46.784916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:46.788389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 352/715] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:46 2022] GET /identity/v3/services/network => generated 88 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:46.789592 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43606ef5-2510-461c-9a44-ddd246c298c9 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:46.789950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43606ef5-2510-461c-9a44-ddd246c298c9 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:46.790313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43606ef5-2510-461c-9a44-ddd246c298c9 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:46.795027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 353/716] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:46.800071 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e424185-2a51-4b94-8c96-976ad8e0fffd None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:46.800741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e424185-2a51-4b94-8c96-976ad8e0fffd None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:46.800741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e424185-2a51-4b94-8c96-976ad8e0fffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:46.828821 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-9e424185-2a51-4b94-8c96-976ad8e0fffd None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:46.830700 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9e424185-2a51-4b94-8c96-976ad8e0fffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:46.884769 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:46.920724 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 354/717] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 13:09:46.921824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e424185-2a51-4b94-8c96-976ad8e0fffd None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:46.936994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db1e2d-9ff2-4fb0-9d99-741e99f8effd None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:46.988939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f5f91-d513-4248-8d08-a5ecd69ee7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:46.989646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f5f91-d513-4248-8d08-a5ecd69ee7ab None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:46.990013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f5f91-d513-4248-8d08-a5ecd69ee7ab None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:46.990263 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f5f91-d513-4248-8d08-a5ecd69ee7ab None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:46.991957 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f5f91-d513-4248-8d08-a5ecd69ee7ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:46.991957 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f5f91-d513-4248-8d08-a5ecd69ee7ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:46.993937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f5f91-d513-4248-8d08-a5ecd69ee7ab None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:47.010260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:47.012938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 364/718] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:09:47.013956 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f5f91-d513-4248-8d08-a5ecd69ee7ab None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:47.065222 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e424185-2a51-4b94-8c96-976ad8e0fffd None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:47.108966 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c709f8e-cd9e-4227-9307-5073cb37059a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:47.110164 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c709f8e-cd9e-4227-9307-5073cb37059a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:47.110449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c709f8e-cd9e-4227-9307-5073cb37059a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:47.110786 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c709f8e-cd9e-4227-9307-5073cb37059a None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:47.111515 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c709f8e-cd9e-4227-9307-5073cb37059a None admin] RBAC: Adding query filter params (name=network) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:47.111819 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c709f8e-cd9e-4227-9307-5073cb37059a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:47.113450 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c709f8e-cd9e-4227-9307-5073cb37059a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:47.123718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 355/719] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 7 13:09:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:47.124734 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c709f8e-cd9e-4227-9307-5073cb37059a None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:47.146024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f5f91-d513-4248-8d08-a5ecd69ee7ab None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:47.181260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:47.195113 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950748a1-66eb-45ab-b352-2569d27227b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:47.196022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950748a1-66eb-45ab-b352-2569d27227b3 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:47.196541 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950748a1-66eb-45ab-b352-2569d27227b3 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:47.196961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950748a1-66eb-45ab-b352-2569d27227b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:47.211189 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950748a1-66eb-45ab-b352-2569d27227b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:47.217992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-950748a1-66eb-45ab-b352-2569d27227b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:47.217992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:47.217992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:47.217992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:47.217992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:47.217992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:47.217992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:47.217992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:47.217992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:47.217992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:47.217992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:47.217992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:47.217992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:47.217992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:47.217992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:47.217992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:47.220454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:47.220454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:47.220454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:47.220454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:47.220454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:47.220454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:47.220454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:47.220454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:47.220454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:47.220454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:47.220454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:47.220454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:47.220454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:47.220454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:47.220454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:47.220454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:47.221713 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:47.221713 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:47.221713 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:47.221713 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:47.221713 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:47.221713 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:47.221713 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:47.221713 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:47.221713 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:47.221713 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:47.221713 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:47.221713 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:47.229166 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 365/720] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:47.230569 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950748a1-66eb-45ab-b352-2569d27227b3 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:47.250943 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c709f8e-cd9e-4227-9307-5073cb37059a None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:47.283374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:47.296007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb323e-18f3-45f3-89fe-1de393ed5e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:47.297010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb323e-18f3-45f3-89fe-1de393ed5e39 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:47.297257 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb323e-18f3-45f3-89fe-1de393ed5e39 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:47.297529 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb323e-18f3-45f3-89fe-1de393ed5e39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:47.306114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb323e-18f3-45f3-89fe-1de393ed5e39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:47.307614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb323e-18f3-45f3-89fe-1de393ed5e39 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:47.319655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-cccb323e-18f3-45f3-89fe-1de393ed5e39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:47.319655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:47.319655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:47.319655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:47.319655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:47.319655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:47.319655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:47.319655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:47.319655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:47.319655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:47.319655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:47.319655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:47.319655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:47.319655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:47.319655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:47.320952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:47.320952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:47.320952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:47.320952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:47.320952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:47.320952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:47.320952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:47.320952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:47.320952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:47.320952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:47.320952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:47.320952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:47.320952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:47.320952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:47.320952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:47.320952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:47.320952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:47.322225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:47.322225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:47.322225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:47.322225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:47.322225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:47.322225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:47.322225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:47.322225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:47.322225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 356/721] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:47.322225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb323e-18f3-45f3-89fe-1de393ed5e39 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:47.377052 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950748a1-66eb-45ab-b352-2569d27227b3 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:47.426072 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300edaf-0ef9-4a60-b954-d81e3d7ce5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:47.427191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300edaf-0ef9-4a60-b954-d81e3d7ce5f2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:47.427551 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300edaf-0ef9-4a60-b954-d81e3d7ce5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:47.427855 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300edaf-0ef9-4a60-b954-d81e3d7ce5f2 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:47.428565 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300edaf-0ef9-4a60-b954-d81e3d7ce5f2 None admin] RBAC: Adding query filter params (type=network) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:47.428867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300edaf-0ef9-4a60-b954-d81e3d7ce5f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:47.430394 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300edaf-0ef9-4a60-b954-d81e3d7ce5f2 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:47.443415 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 366/722] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 7 13:09:47 2022] GET /identity/v3/services?type=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:47.444445 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300edaf-0ef9-4a60-b954-d81e3d7ce5f2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:47.466887 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb323e-18f3-45f3-89fe-1de393ed5e39 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:47.537255 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df1fbc-2a3e-4998-af7a-cc7cc4f89fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:47.538403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df1fbc-2a3e-4998-af7a-cc7cc4f89fad None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:47.538795 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df1fbc-2a3e-4998-af7a-cc7cc4f89fad None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:47.539272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df1fbc-2a3e-4998-af7a-cc7cc4f89fad None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:47.540484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df1fbc-2a3e-4998-af7a-cc7cc4f89fad None admin] RBAC: Adding query filter params (name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:47.540946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df1fbc-2a3e-4998-af7a-cc7cc4f89fad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:47.544350 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df1fbc-2a3e-4998-af7a-cc7cc4f89fad None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:47.561469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:47.568780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 357/723] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:09:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:47.568780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df1fbc-2a3e-4998-af7a-cc7cc4f89fad None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:47.646807 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:47.650176 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300edaf-0ef9-4a60-b954-d81e3d7ce5f2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:47.711271 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01320178-9367-4baa-a128-a56f920afb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:47.712142 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01320178-9367-4baa-a128-a56f920afb20 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:47.712589 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01320178-9367-4baa-a128-a56f920afb20 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:47.713028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01320178-9367-4baa-a128-a56f920afb20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:47.732708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01320178-9367-4baa-a128-a56f920afb20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:47.742415 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df1fbc-2a3e-4998-af7a-cc7cc4f89fad None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:47.743388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-01320178-9367-4baa-a128-a56f920afb20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:09:47.743388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:47.743388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:47.743388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:47.743388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:47.743388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:47.743388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:47.743388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:47.743388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:47.743388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:47.743388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:47.743388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:47.743388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:47.743388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:47.743388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:47.743388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:47.744611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:47.744611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:47.744611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:47.744611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:47.744611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:47.744611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:47.744611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:47.744611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:47.744611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:47.744611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:47.744611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:47.744611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:47.744611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:47.744611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:47.744611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:47.744611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:47.745757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:47.745757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:47.745757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:47.745757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:47.745757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:47.745757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:47.745757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:47.745757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:47.745757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:47.745757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:47.745757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:09:47.745757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:47.745757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 367/724] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:09:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:47.747679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b224bf-78e1-4aa1-ad80-4ceef35e0a8b None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:47.748013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b224bf-78e1-4aa1-ad80-4ceef35e0a8b None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:47.748491 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b224bf-78e1-4aa1-ad80-4ceef35e0a8b None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:47.756755 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 368/725] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:47 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:47.757875 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b224bf-78e1-4aa1-ad80-4ceef35e0a8b None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:47.805688 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921d203-4c40-483a-a665-215e542e86c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:47.806473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921d203-4c40-483a-a665-215e542e86c1 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:47.806608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921d203-4c40-483a-a665-215e542e86c1 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:47.806963 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921d203-4c40-483a-a665-215e542e86c1 None admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:47.807644 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921d203-4c40-483a-a665-215e542e86c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:47.807952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921d203-4c40-483a-a665-215e542e86c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:47.809414 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921d203-4c40-483a-a665-215e542e86c1 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:47.829465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 358/726] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:47.829465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921d203-4c40-483a-a665-215e542e86c1 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:47.908690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b224bf-78e1-4aa1-ad80-4ceef35e0a8b None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:47.920629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:47.950312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82363f-1fb7-4b2a-b075-5ebfaf32fbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:47.951008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82363f-1fb7-4b2a-b075-5ebfaf32fbc6 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:47.951300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82363f-1fb7-4b2a-b075-5ebfaf32fbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:47.951578 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82363f-1fb7-4b2a-b075-5ebfaf32fbc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:47.959987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921d203-4c40-483a-a665-215e542e86c1 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:47.965805 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82363f-1fb7-4b2a-b075-5ebfaf32fbc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:47.967290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82363f-1fb7-4b2a-b075-5ebfaf32fbc6 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:47.968721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:47.986040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-5c82363f-1fb7-4b2a-b075-5ebfaf32fbc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:47.986040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:47.986040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:47.986040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:47.986040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:47.986040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:47.986040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:47.986040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:47.986040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:47.986040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:47.986040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:47.986040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:47.986040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:47.986040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:47.986040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:47.986040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:47.987472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:47.987472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:47.987472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:47.987472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:47.987472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:47.987472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:47.987472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:47.987472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:47.987472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:47.987472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:47.987472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:47.987472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:47.987472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:47.987472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:47.987472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:47.987472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:47.987472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:47.988530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:47.988530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:47.988530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:47.991405 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 369/727] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:47.993197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f840d9b-dc87-48f6-afe7-c22530695298 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:47.993483 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f840d9b-dc87-48f6-afe7-c22530695298 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:47.993764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f840d9b-dc87-48f6-afe7-c22530695298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:48.009821 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7136360-e823-48b1-819a-dcfd8c3deb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:48.010607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7136360-e823-48b1-819a-dcfd8c3deb48 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:48.010727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7136360-e823-48b1-819a-dcfd8c3deb48 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:48.011013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7136360-e823-48b1-819a-dcfd8c3deb48 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:48.011752 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7136360-e823-48b1-819a-dcfd8c3deb48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:48.012065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7136360-e823-48b1-819a-dcfd8c3deb48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:48.013581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7136360-e823-48b1-819a-dcfd8c3deb48 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:48.024772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-4f840d9b-dc87-48f6-afe7-c22530695298 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:48.026337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-4f840d9b-dc87-48f6-afe7-c22530695298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:48.033506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 359/728] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:09:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:48.033506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7136360-e823-48b1-819a-dcfd8c3deb48 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:48.118718 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 370/729] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:48.120043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f840d9b-dc87-48f6-afe7-c22530695298 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:48.167251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7136360-e823-48b1-819a-dcfd8c3deb48 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:48.219519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b532e4-a30d-4cff-99e3-2c957c06fb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:48.220706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b532e4-a30d-4cff-99e3-2c957c06fb69 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:48.223161 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b532e4-a30d-4cff-99e3-2c957c06fb69 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:48.223585 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b532e4-a30d-4cff-99e3-2c957c06fb69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:48.224576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b532e4-a30d-4cff-99e3-2c957c06fb69 None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=97c363046db444cbb3d4346066f9ad42) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:48.224997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b532e4-a30d-4cff-99e3-2c957c06fb69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:48.227386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b532e4-a30d-4cff-99e3-2c957c06fb69 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:48.235048 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 360/730] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:09:48 2022] GET /identity/v3/role_assignments?user.id=97c363046db444cbb3d4346066f9ad42&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:48.236507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b532e4-a30d-4cff-99e3-2c957c06fb69 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:48.250601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:48.266281 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f840d9b-dc87-48f6-afe7-c22530695298 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:48.283015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:48.323044 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3dd19-7ced-439e-9859-4fe691c31126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:48.324103 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3dd19-7ced-439e-9859-4fe691c31126 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:48.324571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3dd19-7ced-439e-9859-4fe691c31126 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:48.325005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3dd19-7ced-439e-9859-4fe691c31126 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:48.325822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3dd19-7ced-439e-9859-4fe691c31126 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:48.329457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3dd19-7ced-439e-9859-4fe691c31126 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:48.334154 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3dd19-7ced-439e-9859-4fe691c31126 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:48.368571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 371/731] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:48.370952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3dd19-7ced-439e-9859-4fe691c31126 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:48.442312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b532e4-a30d-4cff-99e3-2c957c06fb69 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:48.504832 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b5842-6f68-493c-9e01-933a215ec424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:48.505600 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b5842-6f68-493c-9e01-933a215ec424 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:48.505905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b5842-6f68-493c-9e01-933a215ec424 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:48.506300 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b5842-6f68-493c-9e01-933a215ec424 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:48.520745 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b5842-6f68-493c-9e01-933a215ec424 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:48.522430 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b5842-6f68-493c-9e01-933a215ec424 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:48.545975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-034b5842-6f68-493c-9e01-933a215ec424 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:48.545975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:48.545975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:48.545975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:48.545975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:48.545975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:48.545975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:48.545975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:48.545975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:48.545975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:48.545975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:48.545975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:48.545975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:48.545975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:48.545975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:48.545975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:48.548010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:48.548010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:48.548010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:48.548010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:48.548010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:48.548010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:48.548010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:48.548010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:48.548010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:48.548010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:48.548010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:48.548010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:48.548010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:48.548010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:48.548010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:48.548010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:48.548010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:48.548923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:48.548923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:48.548923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:48.548923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 361/732] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:48.548923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b5842-6f68-493c-9e01-933a215ec424 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:48.567957 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3dd19-7ced-439e-9859-4fe691c31126 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:48.623114 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c084958-8b78-4531-8338-9162f6a2f3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:48.624109 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c084958-8b78-4531-8338-9162f6a2f3a8 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:48.624402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c084958-8b78-4531-8338-9162f6a2f3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:48.624677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c084958-8b78-4531-8338-9162f6a2f3a8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:48.636747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c084958-8b78-4531-8338-9162f6a2f3a8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:48.638446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c084958-8b78-4531-8338-9162f6a2f3a8 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:48.655648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-5c084958-8b78-4531-8338-9162f6a2f3a8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:09:48.655648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:48.655648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:48.655648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:48.655648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:48.655648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:48.655648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:48.655648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:48.655648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:48.655648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:48.655648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:48.655648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:48.655648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:48.655648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:48.655648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:48.655648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:48.657337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:48.658732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:48.658732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:48.658732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:48.658732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:48.658732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:48.658732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:48.658732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:48.658732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:48.658732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:09:48.658732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:48.659346 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:48.659459 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 372/733] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 7 13:09:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:09:48.660056 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c084958-8b78-4531-8338-9162f6a2f3a8 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:48.683408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:48.683846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b5842-6f68-493c-9e01-933a215ec424 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:48.733019 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf40fc-c9c6-4731-a50b-28eaa08a54dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:48.733572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf40fc-c9c6-4731-a50b-28eaa08a54dc None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:48.733855 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf40fc-c9c6-4731-a50b-28eaa08a54dc None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:48.734129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf40fc-c9c6-4731-a50b-28eaa08a54dc None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:48.735205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf40fc-c9c6-4731-a50b-28eaa08a54dc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:48.736811 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf40fc-c9c6-4731-a50b-28eaa08a54dc None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:48.797594 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c084958-8b78-4531-8338-9162f6a2f3a8 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:48.841360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72a066-7612-4bba-a04c-9d1963fe3e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:48.842090 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72a066-7612-4bba-a04c-9d1963fe3e16 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:48.842366 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72a066-7612-4bba-a04c-9d1963fe3e16 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:48.842718 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72a066-7612-4bba-a04c-9d1963fe3e16 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:48.843460 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72a066-7612-4bba-a04c-9d1963fe3e16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:48.844163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72a066-7612-4bba-a04c-9d1963fe3e16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:48.845618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72a066-7612-4bba-a04c-9d1963fe3e16 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:48.855972 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 362/734] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 7 13:09:48 2022] POST /identity/v3/services => generated 251 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:09:48.856867 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf40fc-c9c6-4731-a50b-28eaa08a54dc None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:48.878789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 373/735] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:48.887302 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72a066-7612-4bba-a04c-9d1963fe3e16 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:49.004830 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:49.030649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf40fc-c9c6-4731-a50b-28eaa08a54dc None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:49.061850 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72a066-7612-4bba-a04c-9d1963fe3e16 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:49.098531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f366457-48ee-456d-82cc-215a205860be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:49.099618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f366457-48ee-456d-82cc-215a205860be None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:49.100387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f366457-48ee-456d-82cc-215a205860be None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:49.100579 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f366457-48ee-456d-82cc-215a205860be None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:49.101809 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f366457-48ee-456d-82cc-215a205860be None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:49.102375 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f366457-48ee-456d-82cc-215a205860be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:49.105057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f366457-48ee-456d-82cc-215a205860be None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:49.116529 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8346e8-79d6-4023-b725-149b545cef60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:49.117763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8346e8-79d6-4023-b725-149b545cef60 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:49.118326 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8346e8-79d6-4023-b725-149b545cef60 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:49.118968 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8346e8-79d6-4023-b725-149b545cef60 None admin] PATH_INFO: `/v3/users/swift` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:49.130435 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8346e8-79d6-4023-b725-149b545cef60 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:49.132366 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8346e8-79d6-4023-b725-149b545cef60 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:49.146029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:49.151242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-6c8346e8-79d6-4023-b725-149b545cef60 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:09:49.151242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:49.151242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:49.151242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:49.151242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:49.151242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:49.151242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:49.151242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:49.151242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:49.151242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:49.151242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:49.151242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:49.151242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:49.151242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:49.151242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:49.151242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:49.152280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:49.153259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:49.153259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:49.153259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:49.153259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:49.153259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:49.153259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:49.153259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:49.153259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:49.153259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:09:49.153259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:49.157231 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 374/736] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:09:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:09:49.170169 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8346e8-79d6-4023-b725-149b545cef60 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:49.178063 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 363/737] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Sep 7 13:09:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:49.182222 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f366457-48ee-456d-82cc-215a205860be None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:49.409398 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f366457-48ee-456d-82cc-215a205860be None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:49.494507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae8630-7cb9-46e7-93c3-66b69cf9a26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:49.495584 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae8630-7cb9-46e7-93c3-66b69cf9a26b None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:49.496043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae8630-7cb9-46e7-93c3-66b69cf9a26b None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:49.498160 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae8630-7cb9-46e7-93c3-66b69cf9a26b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:49.514235 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8346e8-79d6-4023-b725-149b545cef60 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:49.635023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:49.663440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:49.668566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ae27d-a19c-4345-be84-96cdb742ff17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:49.669214 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ae27d-a19c-4345-be84-96cdb742ff17 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:49.669495 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ae27d-a19c-4345-be84-96cdb742ff17 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:49.669889 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ae27d-a19c-4345-be84-96cdb742ff17 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:49.670070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae8630-7cb9-46e7-93c3-66b69cf9a26b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:49.670668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ae27d-a19c-4345-be84-96cdb742ff17 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:49.671005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ae27d-a19c-4345-be84-96cdb742ff17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:49.672956 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ae27d-a19c-4345-be84-96cdb742ff17 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:49.673125 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae8630-7cb9-46e7-93c3-66b69cf9a26b None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:49.736296 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 375/738] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Sep 7 13:09:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 567 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:49.748173 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ae27d-a19c-4345-be84-96cdb742ff17 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:49.983206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 364/739] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:09:49 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 798 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:09:49.988687 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae8630-7cb9-46e7-93c3-66b69cf9a26b None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:50.082467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ae27d-a19c-4345-be84-96cdb742ff17 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:50.148400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ee5d9-171f-4a54-935f-3e88b69b9a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:50.149724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ee5d9-171f-4a54-935f-3e88b69b9a32 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:50.150389 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ee5d9-171f-4a54-935f-3e88b69b9a32 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:50.150889 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ee5d9-171f-4a54-935f-3e88b69b9a32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:50.191469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:50.201979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae8630-7cb9-46e7-93c3-66b69cf9a26b None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:50.238022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:50.259071 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ee5d9-171f-4a54-935f-3e88b69b9a32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:50.261332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ee5d9-171f-4a54-935f-3e88b69b9a32 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:50.291410 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429fb95b-e908-4aea-9577-7f229cc63122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:50.292737 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429fb95b-e908-4aea-9577-7f229cc63122 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:50.293241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429fb95b-e908-4aea-9577-7f229cc63122 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:50.293753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429fb95b-e908-4aea-9577-7f229cc63122 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:50.308453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429fb95b-e908-4aea-9577-7f229cc63122 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:50.310278 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429fb95b-e908-4aea-9577-7f229cc63122 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:50.334953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-429fb95b-e908-4aea-9577-7f229cc63122 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:50.334953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:50.334953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:50.334953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:50.334953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:50.334953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:50.334953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:50.334953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:50.334953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:50.334953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:50.334953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:50.334953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:50.334953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:50.334953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:50.334953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:50.334953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:50.336007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:50.336007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:50.336007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:50.336007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:50.336007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:50.336007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:50.336007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:50.336007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:50.336007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:50.336007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:50.336007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:50.336007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:50.336007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:50.336007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:50.336007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:50.336007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:50.336007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:50.336953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:50.336953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:50.336953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:50.339711 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 365/740] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:50.341388 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58f403-30e6-4d6f-bceb-9bbbee3227c9 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:50.341820 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58f403-30e6-4d6f-bceb-9bbbee3227c9 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:50.342320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58f403-30e6-4d6f-bceb-9bbbee3227c9 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:50.349035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 366/741] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:50.351770 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62165864-8360-4c38-8ea8-f581b0cfa9a3 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:50.352264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62165864-8360-4c38-8ea8-f581b0cfa9a3 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:50.352794 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62165864-8360-4c38-8ea8-f581b0cfa9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:50.378194 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-62165864-8360-4c38-8ea8-f581b0cfa9a3 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:50.380489 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-62165864-8360-4c38-8ea8-f581b0cfa9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:50.486687 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 376/742] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:09:49 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 13:09:50.487866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ee5d9-171f-4a54-935f-3e88b69b9a32 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:50.516293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 367/743] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 13:09:50.518174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62165864-8360-4c38-8ea8-f581b0cfa9a3 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:50.659552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:50.662907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:50.674711 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ee5d9-171f-4a54-935f-3e88b69b9a32 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:50.683591 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62165864-8360-4c38-8ea8-f581b0cfa9a3 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:50.711086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c20b4-a5bb-43b1-8d08-d212ecd68341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:50.711786 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c20b4-a5bb-43b1-8d08-d212ecd68341 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:50.712074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c20b4-a5bb-43b1-8d08-d212ecd68341 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:50.712350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c20b4-a5bb-43b1-8d08-d212ecd68341 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:50.713013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c20b4-a5bb-43b1-8d08-d212ecd68341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:50.713373 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c20b4-a5bb-43b1-8d08-d212ecd68341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:50.714820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c20b4-a5bb-43b1-8d08-d212ecd68341 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:50.724592 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85edb0-78d6-41e6-86b3-2595a8ef6b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:50.725606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85edb0-78d6-41e6-86b3-2595a8ef6b59 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:50.726081 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85edb0-78d6-41e6-86b3-2595a8ef6b59 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:50.726691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85edb0-78d6-41e6-86b3-2595a8ef6b59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:50.729198 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 377/744] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:50.741655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c20b4-a5bb-43b1-8d08-d212ecd68341 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:50.744853 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a85edb0-78d6-41e6-86b3-2595a8ef6b59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:50.746531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a85edb0-78d6-41e6-86b3-2595a8ef6b59 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:50.777323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-6a85edb0-78d6-41e6-86b3-2595a8ef6b59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:50.777323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:50.777323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:50.777323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:50.777323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:50.777323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:50.777323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:50.777323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:50.777323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:50.777323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:50.777323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:50.777323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:50.777323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:50.777323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:50.777323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:50.777323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:50.778331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:50.778331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:50.778331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:50.778331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:50.778331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:50.778331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:50.778331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:50.778331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:50.778331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:50.778331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:50.778331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:50.778331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:50.778331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:50.778331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:50.778331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:50.778331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:50.778331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:50.779256 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:50.779256 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:50.779256 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:50.783296 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 368/745] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:50.792024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85edb0-78d6-41e6-86b3-2595a8ef6b59 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:51.016878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c20b4-a5bb-43b1-8d08-d212ecd68341 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:51.048514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85edb0-78d6-41e6-86b3-2595a8ef6b59 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:51.096115 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c69e7b-8a91-403f-9df9-cfc36d46660c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:51.097670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c69e7b-8a91-403f-9df9-cfc36d46660c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:51.098516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c69e7b-8a91-403f-9df9-cfc36d46660c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:51.099227 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c69e7b-8a91-403f-9df9-cfc36d46660c None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:51.111237 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:51.116199 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c69e7b-8a91-403f-9df9-cfc36d46660c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:51.117928 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45724e-b7c6-4253-8dd6-ad96ddd7f2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:51.118676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45724e-b7c6-4253-8dd6-ad96ddd7f2d9 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:51.119049 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45724e-b7c6-4253-8dd6-ad96ddd7f2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:51.123638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45724e-b7c6-4253-8dd6-ad96ddd7f2d9 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:51.123638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45724e-b7c6-4253-8dd6-ad96ddd7f2d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:51.123969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c69e7b-8a91-403f-9df9-cfc36d46660c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:51.124125 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45724e-b7c6-4253-8dd6-ad96ddd7f2d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:51.124125 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45724e-b7c6-4253-8dd6-ad96ddd7f2d9 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:51.139389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 369/746] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:51.141682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45724e-b7c6-4253-8dd6-ad96ddd7f2d9 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:51.147063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-69c69e7b-8a91-403f-9df9-cfc36d46660c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:51.147063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:51.147063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:51.147063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:51.147063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:51.147063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:51.147063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:51.147063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:51.147063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:51.147063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:51.147063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:51.147063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:51.147063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:51.147063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:51.147063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:51.148739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:51.148739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:51.148739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:51.148739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:51.148739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:51.148739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:51.148739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:51.148739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:51.148739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:51.148739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:51.148739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:51.148739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:51.148739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:51.148739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:51.148739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:51.148739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:51.148739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:51.150679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:51.150679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:51.150679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:51.150679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:51.150679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:51.150679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:51.150679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:51.150679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:51.160359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 378/747] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:09:51.161951 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c69e7b-8a91-403f-9df9-cfc36d46660c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:51.183256 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:51.351880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45724e-b7c6-4253-8dd6-ad96ddd7f2d9 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:51.359250 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c69e7b-8a91-403f-9df9-cfc36d46660c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:51.421228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1449f-0d35-49f8-a492-f2065fcc19b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:51.423746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1449f-0d35-49f8-a492-f2065fcc19b5 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:51.424823 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1449f-0d35-49f8-a492-f2065fcc19b5 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:51.425873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1449f-0d35-49f8-a492-f2065fcc19b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:51.427725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec0743-d7a4-4585-859a-736652108f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:51.428370 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec0743-d7a4-4585-859a-736652108f03 None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:51.428678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec0743-d7a4-4585-859a-736652108f03 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:51.428954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec0743-d7a4-4585-859a-736652108f03 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:51.429675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec0743-d7a4-4585-859a-736652108f03 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:51.431459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec0743-d7a4-4585-859a-736652108f03 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:51.444268 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1449f-0d35-49f8-a492-f2065fcc19b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:51.445816 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed1449f-0d35-49f8-a492-f2065fcc19b5 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:51.471531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-aed1449f-0d35-49f8-a492-f2065fcc19b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:51.471531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:51.471531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:51.471531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:51.471531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:51.471531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:51.471531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:51.471531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:51.471531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:51.471531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:51.471531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:51.471531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:51.471531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:51.471531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:51.471531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:51.472758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:51.472758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:51.472758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:51.472758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:51.472758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:51.472758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:51.472758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:51.472758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:51.472758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:51.472758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:51.472758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:51.472758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:51.472758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:51.472758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:51.472758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:51.472758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:51.472758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:51.473712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:51.473712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:51.473712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:51.473712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:51.473712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:51.473712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:51.473712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:51.473712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:51.473712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 379/748] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:51.478503 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9ffb4-fc34-4a1b-9bfa-10c023b7f779 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:51.478503 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9ffb4-fc34-4a1b-9bfa-10c023b7f779 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:51.478503 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9ffb4-fc34-4a1b-9bfa-10c023b7f779 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:51.479947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 380/749] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:51.483187 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9ffb4-fc34-4a1b-9bfa-10c023b7f779 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:51.567921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 370/750] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 7 13:09:51 2022] POST /identity/v3/services => generated 240 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:09:51.569896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec0743-d7a4-4585-859a-736652108f03 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:51.624308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:51.695607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:51.751688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9ffb4-fc34-4a1b-9bfa-10c023b7f779 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:51.816114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec0743-d7a4-4585-859a-736652108f03 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:51.832643 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd89f3-367a-4488-bc4d-dd42dd812d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:51.833348 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd89f3-367a-4488-bc4d-dd42dd812d68 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:51.833631 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd89f3-367a-4488-bc4d-dd42dd812d68 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:51.833907 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd89f3-367a-4488-bc4d-dd42dd812d68 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:51.834721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd89f3-367a-4488-bc4d-dd42dd812d68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:51.835100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd89f3-367a-4488-bc4d-dd42dd812d68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:51.837328 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd89f3-367a-4488-bc4d-dd42dd812d68 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:51.861486 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 381/751] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:51.866281 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ec66b-72c2-408b-b141-5f35b271719e None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:51.867312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ec66b-72c2-408b-b141-5f35b271719e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:51.868169 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ec66b-72c2-408b-b141-5f35b271719e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:51.904186 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-833ec66b-72c2-408b-b141-5f35b271719e None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:51.906789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-833ec66b-72c2-408b-b141-5f35b271719e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:51.907784 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6011185-9c17-49dd-b900-a45bca6e2d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:51.908076 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6011185-9c17-49dd-b900-a45bca6e2d99 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:51.908569 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6011185-9c17-49dd-b900-a45bca6e2d99 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:51.909059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6011185-9c17-49dd-b900-a45bca6e2d99 None admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:51.910421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6011185-9c17-49dd-b900-a45bca6e2d99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:51.910963 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6011185-9c17-49dd-b900-a45bca6e2d99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:51.913911 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6011185-9c17-49dd-b900-a45bca6e2d99 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:51.940095 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 371/752] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:51.941603 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6011185-9c17-49dd-b900-a45bca6e2d99 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:52.039000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 382/753] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:09:52.040488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ec66b-72c2-408b-b141-5f35b271719e None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:52.168005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:52.246306 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6011185-9c17-49dd-b900-a45bca6e2d99 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:52.264425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ec66b-72c2-408b-b141-5f35b271719e None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:52.298496 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:52.338935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548bc4f-aaa6-47ee-9cce-916147d0626e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:52.340030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548bc4f-aaa6-47ee-9cce-916147d0626e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:52.340475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548bc4f-aaa6-47ee-9cce-916147d0626e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:52.340920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548bc4f-aaa6-47ee-9cce-916147d0626e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:52.342217 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548bc4f-aaa6-47ee-9cce-916147d0626e None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=5cedc7091d2f46b5be68818d26399e9b) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:52.342752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548bc4f-aaa6-47ee-9cce-916147d0626e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:52.345752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548bc4f-aaa6-47ee-9cce-916147d0626e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:52.357082 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 383/754] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:09:52 2022] GET /identity/v3/role_assignments?user.id=5cedc7091d2f46b5be68818d26399e9b&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 259 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:52.391999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd4798-b2a0-484f-834f-5b261dcf2758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:52.392537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021ed7c-19b2-47cf-9557-42551e27bfdf None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:52.393004 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd4798-b2a0-484f-834f-5b261dcf2758 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:52.393162 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021ed7c-19b2-47cf-9557-42551e27bfdf None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:52.393441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021ed7c-19b2-47cf-9557-42551e27bfdf None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:52.393613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd4798-b2a0-484f-834f-5b261dcf2758 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:52.393872 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd4798-b2a0-484f-834f-5b261dcf2758 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:52.395230 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd4798-b2a0-484f-834f-5b261dcf2758 None admin] RBAC: Adding query filter params (role.id=743ed2d8e7114975bd9245742a41fe0f, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=8121ba4cf2cf44d9a5a55975ea85928c) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:52.395742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd4798-b2a0-484f-834f-5b261dcf2758 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:52.396485 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 384/755] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:09:52.400643 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd4798-b2a0-484f-834f-5b261dcf2758 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:52.406366 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e23dd-4d09-48bc-801e-53b534041ba7 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:52.406790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e23dd-4d09-48bc-801e-53b534041ba7 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:52.407300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e23dd-4d09-48bc-801e-53b534041ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:52.418536 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 372/756] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:09:51 2022] GET /identity/v3/role_assignments?user.id=8121ba4cf2cf44d9a5a55975ea85928c&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=743ed2d8e7114975bd9245742a41fe0f => generated 259 bytes in 478 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:52.442031 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-011e23dd-4d09-48bc-801e-53b534041ba7 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:52.444226 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-011e23dd-4d09-48bc-801e-53b534041ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:52.587544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 385/757] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:09:52.719554 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd4798-b2a0-484f-834f-5b261dcf2758 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:52.938889 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd4798-b2a0-484f-834f-5b261dcf2758 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:53.017180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230cef7-a722-4400-9748-235f4d8e8c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:53.017424 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e23dd-4d09-48bc-801e-53b534041ba7 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:53.018337 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230cef7-a722-4400-9748-235f4d8e8c01 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:53.018645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230cef7-a722-4400-9748-235f4d8e8c01 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:53.019154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230cef7-a722-4400-9748-235f4d8e8c01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:53.051506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230cef7-a722-4400-9748-235f4d8e8c01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:53.074729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-7230cef7-a722-4400-9748-235f4d8e8c01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:09:53.074729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:53.074729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:53.074729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:53.074729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:53.074729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:53.074729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:53.074729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:53.074729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:53.074729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:53.074729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:53.074729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:53.074729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:53.074729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:53.074729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:53.074729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:53.076636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:53.076636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:53.076636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:53.076636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:53.076636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:53.076636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:53.076636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:53.076636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:53.076636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:53.076636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:53.076636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:53.076636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:53.076636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:53.076636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:53.076636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:53.076636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:53.078396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:53.078396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:53.078396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:53.078396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:53.078396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:53.078396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:53.078396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:53.078396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:53.078396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:53.078396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:53.078396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:09:53.078396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:53.081879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 373/758] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:09:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:53.091815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:53.093094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230cef7-a722-4400-9748-235f4d8e8c01 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:53.189844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e23dd-4d09-48bc-801e-53b534041ba7 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:53.221627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:53.263493 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c4bec-62b8-49c9-bc46-fa00dcb4bf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:53.264673 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c4bec-62b8-49c9-bc46-fa00dcb4bf8b None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:53.265995 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c4bec-62b8-49c9-bc46-fa00dcb4bf8b None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:53.269519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c4bec-62b8-49c9-bc46-fa00dcb4bf8b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:53.270770 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c4bec-62b8-49c9-bc46-fa00dcb4bf8b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:53.272941 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c4bec-62b8-49c9-bc46-fa00dcb4bf8b None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:53.290426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-a41c4bec-62b8-49c9-bc46-fa00dcb4bf8b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:09:53.290426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:53.290426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:09:53.290426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:09:53.290426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:09:53.290426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:09:53.290426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:09:53.290426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:09:53.290426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:09:53.290426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:53.290426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:09:53.290426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:53.290426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:53.290426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:53.290426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:53.290426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:53.290426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:53.292301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:53.292301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:53.292301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:53.292301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:53.292301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:53.292301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:53.292301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:09:53.292301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:09:53.292301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:09:53.292301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:09:53.292301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:53.292301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:53.292301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:53.292301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:53.292301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:53.292301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:53.293451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:53.293451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:53.293451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:53.293451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:53.293451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:53.293451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:53.293451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:53.293451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:53.293451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:53.293451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:53.293451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:09:53.293451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:09:53.293451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:09:53.293451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:53.298159 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 386/759] 10.222.0.62 () {56 vars in 1129 bytes} [Wed Sep 7 13:09:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:53.305430 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c4bec-62b8-49c9-bc46-fa00dcb4bf8b None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:53.434831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230cef7-a722-4400-9748-235f4d8e8c01 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:53.565346 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7706bd-ea9f-47a6-af39-00d93218912c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:53.566854 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7706bd-ea9f-47a6-af39-00d93218912c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:53.567512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c4bec-62b8-49c9-bc46-fa00dcb4bf8b None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:53.567999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7706bd-ea9f-47a6-af39-00d93218912c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:53.568729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7706bd-ea9f-47a6-af39-00d93218912c None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:53.570192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7706bd-ea9f-47a6-af39-00d93218912c None admin] RBAC: Adding query filter params (name=admin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:53.571068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7706bd-ea9f-47a6-af39-00d93218912c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:53.573861 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7706bd-ea9f-47a6-af39-00d93218912c None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:53.599018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:53.603561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 374/760] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:09:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 512 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:09:53.623110 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7706bd-ea9f-47a6-af39-00d93218912c None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:53.664545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47396f-5156-48d2-a458-743e0bc5d1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:53.665336 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47396f-5156-48d2-a458-743e0bc5d1b5 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:53.665882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47396f-5156-48d2-a458-743e0bc5d1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:53.666343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47396f-5156-48d2-a458-743e0bc5d1b5 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:53.667663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47396f-5156-48d2-a458-743e0bc5d1b5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:53.668204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47396f-5156-48d2-a458-743e0bc5d1b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:53.671242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47396f-5156-48d2-a458-743e0bc5d1b5 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:53.691618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 387/761] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Sep 7 13:09:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:09:53.701559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47396f-5156-48d2-a458-743e0bc5d1b5 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:53.924848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47396f-5156-48d2-a458-743e0bc5d1b5 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:53.958861 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:53.970904 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7706bd-ea9f-47a6-af39-00d93218912c None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:54.032427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed84f23-258d-47b0-80a5-dd2117d35e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:54.033964 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed84f23-258d-47b0-80a5-dd2117d35e6b None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:54.034461 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed84f23-258d-47b0-80a5-dd2117d35e6b None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:54.034945 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed84f23-258d-47b0-80a5-dd2117d35e6b None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:54.035952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed84f23-258d-47b0-80a5-dd2117d35e6b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:54.036768 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed84f23-258d-47b0-80a5-dd2117d35e6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:54.039094 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed84f23-258d-47b0-80a5-dd2117d35e6b None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:54.050389 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 388/762] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Sep 7 13:09:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:09:54.061256 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed84f23-258d-47b0-80a5-dd2117d35e6b None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:54.065361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa909af0-610c-4874-b7bf-2782b197b870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:54.066409 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa909af0-610c-4874-b7bf-2782b197b870 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:54.066880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa909af0-610c-4874-b7bf-2782b197b870 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:54.067365 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa909af0-610c-4874-b7bf-2782b197b870 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:54.108408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa909af0-610c-4874-b7bf-2782b197b870 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:54.111236 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa909af0-610c-4874-b7bf-2782b197b870 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:54.151520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-fa909af0-610c-4874-b7bf-2782b197b870 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:54.151520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:54.151520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:54.151520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:54.151520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:54.151520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:54.151520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:54.151520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:54.151520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:54.151520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:54.151520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:54.151520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:54.151520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:54.151520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:54.151520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:54.151520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:54.152761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:54.152761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:54.152761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:54.152761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:54.152761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:54.152761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:54.152761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:54.152761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:54.152761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:54.152761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:54.152761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:54.152761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:54.152761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:54.152761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:54.152761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:54.152761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:54.152761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:54.153678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:54.153678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:54.153678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:54.163410 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 375/763] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 541 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:54.167530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:54.175102 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa909af0-610c-4874-b7bf-2782b197b870 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:54.309751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed84f23-258d-47b0-80a5-dd2117d35e6b None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:54.391873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07ba69-fe22-40dc-ab9b-9b37398ac5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:54.392729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07ba69-fe22-40dc-ab9b-9b37398ac5ac None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:54.393030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07ba69-fe22-40dc-ab9b-9b37398ac5ac None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:54.393304 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07ba69-fe22-40dc-ab9b-9b37398ac5ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:54.394153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07ba69-fe22-40dc-ab9b-9b37398ac5ac None admin] RBAC: Adding query filter params (interface=public, service_id=29c80d3cfd6c4d4294b661902daf674b, region_id=RegionOne) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:54.394470 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07ba69-fe22-40dc-ab9b-9b37398ac5ac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:54.396109 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07ba69-fe22-40dc-ab9b-9b37398ac5ac None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:54.415176 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 389/764] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Sep 7 13:09:54 2022] GET /identity/v3/endpoints?service_id=29c80d3cfd6c4d4294b661902daf674b&interface=public®ion_id=RegionOne => generated 194 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:09:54.440150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa909af0-610c-4874-b7bf-2782b197b870 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:54.443879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07ba69-fe22-40dc-ab9b-9b37398ac5ac None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:54.526014 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a4ced-9b46-4ff8-9c99-d973f6d1c91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:54.527070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a4ced-9b46-4ff8-9c99-d973f6d1c91b None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:54.527804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a4ced-9b46-4ff8-9c99-d973f6d1c91b None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:54.528292 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a4ced-9b46-4ff8-9c99-d973f6d1c91b None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:54.529473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a4ced-9b46-4ff8-9c99-d973f6d1c91b None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:54.530002 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a4ced-9b46-4ff8-9c99-d973f6d1c91b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:54.539551 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a4ced-9b46-4ff8-9c99-d973f6d1c91b None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:54.579900 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 376/765] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:54.589310 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a4ced-9b46-4ff8-9c99-d973f6d1c91b None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:54.637096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:54.873529 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a4ced-9b46-4ff8-9c99-d973f6d1c91b None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:54.954038 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07ba69-fe22-40dc-ab9b-9b37398ac5ac None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:54.959572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:55.011108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd344b-58e1-4c4a-9ddd-eccb9a631c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:55.012223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd344b-58e1-4c4a-9ddd-eccb9a631c82 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:55.012678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd344b-58e1-4c4a-9ddd-eccb9a631c82 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:55.019718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd344b-58e1-4c4a-9ddd-eccb9a631c82 None admin] PATH_INFO: `/v3/users/nova` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:55.040790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd344b-58e1-4c4a-9ddd-eccb9a631c82 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:55.042816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd344b-58e1-4c4a-9ddd-eccb9a631c82 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:55.068914 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcef7e8-6206-4c12-9476-1e3b81daf1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:55.070567 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcef7e8-6206-4c12-9476-1e3b81daf1c4 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:55.071093 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcef7e8-6206-4c12-9476-1e3b81daf1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:55.071574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcef7e8-6206-4c12-9476-1e3b81daf1c4 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:55.072752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcef7e8-6206-4c12-9476-1e3b81daf1c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:55.075351 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcef7e8-6206-4c12-9476-1e3b81daf1c4 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:55.091828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-f1dd344b-58e1-4c4a-9ddd-eccb9a631c82 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:09:55.091828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:55.091828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:55.091828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:55.091828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:55.091828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:55.091828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:55.091828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:55.091828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:55.091828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:55.091828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:55.091828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:55.091828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:55.091828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:55.091828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:55.091828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:55.093027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:55.094731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:55.094731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:55.094731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:55.094731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:55.094731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:55.094731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:55.094731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:55.094731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:55.094731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:09:55.094731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:55.094731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 377/766] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 7 13:09:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 506 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:09:55.099900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 390/767] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 7 13:09:54 2022] GET /identity/v3/services => generated 772 bytes in 656 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:55.115949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd344b-58e1-4c4a-9ddd-eccb9a631c82 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:55.281913 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:55.349758 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd344b-58e1-4c4a-9ddd-eccb9a631c82 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:55.426417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb656d-27ac-4343-ab11-64b2992e9561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:55.427368 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb656d-27ac-4343-ab11-64b2992e9561 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:55.427774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb656d-27ac-4343-ab11-64b2992e9561 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:55.428176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb656d-27ac-4343-ab11-64b2992e9561 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:55.429313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb656d-27ac-4343-ab11-64b2992e9561 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:55.429779 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb656d-27ac-4343-ab11-64b2992e9561 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:55.432742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb656d-27ac-4343-ab11-64b2992e9561 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:55.646632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 378/768] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 7 13:09:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 531 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:09:55.655074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcef7e8-6206-4c12-9476-1e3b81daf1c4 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:55.852587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff20915-5c85-422a-a02e-47b7d351d7ca None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:55.852851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff20915-5c85-422a-a02e-47b7d351d7ca None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:55.853197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff20915-5c85-422a-a02e-47b7d351d7ca None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:55.856880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 379/769] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:55.868422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9dbe4-0d38-4e6a-824c-44318f033491 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:55.869006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9dbe4-0d38-4e6a-824c-44318f033491 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:55.869281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9dbe4-0d38-4e6a-824c-44318f033491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:55.876193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcef7e8-6206-4c12-9476-1e3b81daf1c4 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:55.924904 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-b3c9dbe4-0d38-4e6a-824c-44318f033491 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:55.926622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b3c9dbe4-0d38-4e6a-824c-44318f033491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:55.952345 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d389395-9b2c-4d78-a011-00845e33a516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:55.953493 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d389395-9b2c-4d78-a011-00845e33a516 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:55.954377 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d389395-9b2c-4d78-a011-00845e33a516 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:55.954831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d389395-9b2c-4d78-a011-00845e33a516 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:56.002709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d389395-9b2c-4d78-a011-00845e33a516 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:56.005575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d389395-9b2c-4d78-a011-00845e33a516 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:56.058463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-0d389395-9b2c-4d78-a011-00845e33a516 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:56.058463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:56.058463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:56.058463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:56.058463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:56.058463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:56.058463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:56.058463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:56.058463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:56.058463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:56.058463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:56.058463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:56.058463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:56.058463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:56.058463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:56.058463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:56.059603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:56.059603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:56.059603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:56.059603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:56.059603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:56.059603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:56.059603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:56.059603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:56.059603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:56.059603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:56.059603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:56.059603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:56.059603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:56.059603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:56.059603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:56.059603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:56.059603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:56.060518 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:56.060518 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:56.060518 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:56.069382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 391/770] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:56.084165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d389395-9b2c-4d78-a011-00845e33a516 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:56.128040 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 380/771] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:09:56.139629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:56.264178 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d389395-9b2c-4d78-a011-00845e33a516 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:56.336285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34435f2e-94f8-4a66-aa7e-cef72fd8e9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:56.337342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34435f2e-94f8-4a66-aa7e-cef72fd8e9f6 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:56.337831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34435f2e-94f8-4a66-aa7e-cef72fd8e9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:56.338322 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34435f2e-94f8-4a66-aa7e-cef72fd8e9f6 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:56.339602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34435f2e-94f8-4a66-aa7e-cef72fd8e9f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:56.340153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34435f2e-94f8-4a66-aa7e-cef72fd8e9f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:56.342882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34435f2e-94f8-4a66-aa7e-cef72fd8e9f6 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:56.368550 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 392/772] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:56.378072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9dbe4-0d38-4e6a-824c-44318f033491 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:56.516787 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:56.623678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9dbe4-0d38-4e6a-824c-44318f033491 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:56.723696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e6642-581a-475c-abc4-ee5eb695bab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:56.724665 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e6642-581a-475c-abc4-ee5eb695bab6 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:56.724804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e6642-581a-475c-abc4-ee5eb695bab6 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:56.725297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e6642-581a-475c-abc4-ee5eb695bab6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:56.747735 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e6642-581a-475c-abc4-ee5eb695bab6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:56.750716 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e6642-581a-475c-abc4-ee5eb695bab6 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:56.780094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-7f5e6642-581a-475c-abc4-ee5eb695bab6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:56.780094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:56.780094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:56.780094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:56.780094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:56.780094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:56.780094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:56.780094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:56.780094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:56.780094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:56.780094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:56.780094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:56.780094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:09:56.780094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:09:56.780094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:09:56.783113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:09:56.783113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:56.783113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:56.783113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:56.783113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:56.783113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:56.783113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:56.783113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:56.783113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:56.783113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:56.783113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:56.783113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:56.783113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:56.783113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:56.783113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:56.783113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:56.783113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:56.784902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:09:56.784902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:09:56.784902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:09:56.784902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:09:56.784902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:09:56.784902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:09:56.784902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:09:56.784902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:56.784902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 381/773] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:56.792696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34435f2e-94f8-4a66-aa7e-cef72fd8e9f6 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:56.795766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e6642-581a-475c-abc4-ee5eb695bab6 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:56.994833 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:57.013859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e6642-581a-475c-abc4-ee5eb695bab6 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:57.030247 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34435f2e-94f8-4a66-aa7e-cef72fd8e9f6 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:57.052387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:57.085470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13109825-991e-4c60-9843-1f94030839b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:57.087091 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13109825-991e-4c60-9843-1f94030839b9 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:57.088047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13109825-991e-4c60-9843-1f94030839b9 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:57.088615 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13109825-991e-4c60-9843-1f94030839b9 None admin] PATH_INFO: `/v3/services/network` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:57.090160 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13109825-991e-4c60-9843-1f94030839b9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:57.092913 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13109825-991e-4c60-9843-1f94030839b9 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:57.106111 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc789b7-eaae-4959-93c6-ae71a2458254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:57.106746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-13109825-991e-4c60-9843-1f94030839b9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:09:57.106746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:57.106746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:09:57.106746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:09:57.106746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:09:57.106746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:09:57.106746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:09:57.106746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:09:57.106746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:09:57.106746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:57.106746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:09:57.106746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:57.106746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:57.106746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:57.106746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:57.106746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:57.106746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:57.108595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:57.108595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:57.108595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:57.108595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:57.108595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:57.108595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:57.108595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:09:57.108595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:09:57.108595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:09:57.108595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:09:57.108595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:57.108595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:57.108595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:57.108595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:57.108595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:57.108595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:57.110107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:57.110107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:57.110107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:57.110107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:57.110107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:57.110107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:57.110107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:57.110107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:57.110107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:57.110107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:57.110107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:09:57.110107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:09:57.110107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:09:57.110107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:57.112931 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc789b7-eaae-4959-93c6-ae71a2458254 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:57.112931 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc789b7-eaae-4959-93c6-ae71a2458254 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:57.112931 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc789b7-eaae-4959-93c6-ae71a2458254 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:57.113542 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 382/774] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:09:56 2022] GET /identity/v3/services/network => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:57.113542 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af76767-7439-48e7-ab7f-fd6c53711095 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:57.113542 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af76767-7439-48e7-ab7f-fd6c53711095 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:57.113542 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af76767-7439-48e7-ab7f-fd6c53711095 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:57.120802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 383/775] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:57.121329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc789b7-eaae-4959-93c6-ae71a2458254 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:57.121329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc789b7-eaae-4959-93c6-ae71a2458254 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:57.129528 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc789b7-eaae-4959-93c6-ae71a2458254 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:57.133600 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c7518-9c68-4e9b-bfc1-1c505c99c417 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:57.134088 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c7518-9c68-4e9b-bfc1-1c505c99c417 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:57.134586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c7518-9c68-4e9b-bfc1-1c505c99c417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:57.164270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 393/776] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:09:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:57.166109 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc789b7-eaae-4959-93c6-ae71a2458254 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:57.168343 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-592c7518-9c68-4e9b-bfc1-1c505c99c417 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:57.170482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-592c7518-9c68-4e9b-bfc1-1c505c99c417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:57.315860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 384/777] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:09:57.317466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c7518-9c68-4e9b-bfc1-1c505c99c417 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:57.381397 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc789b7-eaae-4959-93c6-ae71a2458254 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:57.460722 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd12f8-586b-40a1-8f9a-6e45e3c19fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:57.461707 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd12f8-586b-40a1-8f9a-6e45e3c19fb4 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:57.462168 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd12f8-586b-40a1-8f9a-6e45e3c19fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:57.462634 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd12f8-586b-40a1-8f9a-6e45e3c19fb4 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:57.464109 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd12f8-586b-40a1-8f9a-6e45e3c19fb4 None admin] RBAC: Adding query filter params (name=network) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:57.464637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd12f8-586b-40a1-8f9a-6e45e3c19fb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:57.467596 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd12f8-586b-40a1-8f9a-6e45e3c19fb4 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:57.494764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 394/778] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 7 13:09:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:57.497216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dfeef-201d-4250-ae01-1424bda7ba23 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:57.497747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dfeef-201d-4250-ae01-1424bda7ba23 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:57.498772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dfeef-201d-4250-ae01-1424bda7ba23 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:57.501081 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 395/779] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:57.503005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133dfeef-201d-4250-ae01-1424bda7ba23 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:57.510812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:57.552666 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c7518-9c68-4e9b-bfc1-1c505c99c417 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:57.582247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:57.623114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add13c10-e0e7-4163-831e-dbb7c00eafe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:57.624447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add13c10-e0e7-4163-831e-dbb7c00eafe6 None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:57.626148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add13c10-e0e7-4163-831e-dbb7c00eafe6 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:57.626647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add13c10-e0e7-4163-831e-dbb7c00eafe6 None admin] PATH_INFO: `/v3/projects/4574cf3ff270485faf60594019f7e05c/users/97c363046db444cbb3d4346066f9ad42/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:57.629396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add13c10-e0e7-4163-831e-dbb7c00eafe6 None admin] RBAC: Authorizing `identity:create_grant(project_id=4574cf3ff270485faf60594019f7e05c, user_id=97c363046db444cbb3d4346066f9ad42, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:57.632820 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add13c10-e0e7-4163-831e-dbb7c00eafe6 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:57.720792 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 385/780] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:09:57 2022] PUT /identity/v3/projects/4574cf3ff270485faf60594019f7e05c/users/97c363046db444cbb3d4346066f9ad42/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:09:57.720792 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5aff7f-4b7b-4af4-85b1-f94bc6f380b8 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:57.720792 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5aff7f-4b7b-4af4-85b1-f94bc6f380b8 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:57.720792 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5aff7f-4b7b-4af4-85b1-f94bc6f380b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:57.741703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-9f5aff7f-4b7b-4af4-85b1-f94bc6f380b8 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:57.743007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9f5aff7f-4b7b-4af4-85b1-f94bc6f380b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:57.751143 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133dfeef-201d-4250-ae01-1424bda7ba23 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:57.842998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafb457-8aa9-472a-8ef7-2bb9f6f3ce62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:57.844136 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafb457-8aa9-472a-8ef7-2bb9f6f3ce62 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:57.844637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafb457-8aa9-472a-8ef7-2bb9f6f3ce62 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:57.845126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafb457-8aa9-472a-8ef7-2bb9f6f3ce62 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:57.846389 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccafb457-8aa9-472a-8ef7-2bb9f6f3ce62 None admin] RBAC: Adding query filter params (type=network) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:57.846936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccafb457-8aa9-472a-8ef7-2bb9f6f3ce62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:57.850461 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccafb457-8aa9-472a-8ef7-2bb9f6f3ce62 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:57.868340 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 396/781] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 7 13:09:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:57.870453 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafb457-8aa9-472a-8ef7-2bb9f6f3ce62 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:57.879410 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 386/782] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:09:57.880742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5aff7f-4b7b-4af4-85b1-f94bc6f380b8 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:58.016105 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5aff7f-4b7b-4af4-85b1-f94bc6f380b8 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:58.016505 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:58.018642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:58.060017 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65972672-642a-479c-9803-51b97829e9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:58.061107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65972672-642a-479c-9803-51b97829e9ba None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:58.061613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65972672-642a-479c-9803-51b97829e9ba None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:58.062350 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65972672-642a-479c-9803-51b97829e9ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:58.063376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65972672-642a-479c-9803-51b97829e9ba None admin] RBAC: Adding query filter params (interface=public, service_id=d2d1712355744200b0f7fb722abc84ea, region_id=RegionOne) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:58.063953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65972672-642a-479c-9803-51b97829e9ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:58.065660 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65972672-642a-479c-9803-51b97829e9ba None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:58.075219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 387/783] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Sep 7 13:09:57 2022] GET /identity/v3/endpoints?service_id=d2d1712355744200b0f7fb722abc84ea&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:58.088474 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65972672-642a-479c-9803-51b97829e9ba None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:58.095494 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafb457-8aa9-472a-8ef7-2bb9f6f3ce62 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:58.180249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dacb72-f70c-4044-af6e-2e3951cb014f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:58.181383 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dacb72-f70c-4044-af6e-2e3951cb014f None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:58.181778 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dacb72-f70c-4044-af6e-2e3951cb014f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:58.182213 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dacb72-f70c-4044-af6e-2e3951cb014f None admin] PATH_INFO: `/v3/roles/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:58.207252 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dacb72-f70c-4044-af6e-2e3951cb014f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:58.221310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-56dacb72-f70c-4044-af6e-2e3951cb014f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:58.221310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:58.221310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:58.221310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:58.221310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:58.221310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:58.221310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:58.221310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:58.221310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:58.221310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:58.221310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:58.221310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:58.221310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:58.221310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:58.221310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:58.221310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:58.223606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:58.223606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:58.223606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:58.223606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:58.223606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:58.223606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:58.223606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:58.223606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:58.223606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:58.223606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:58.223606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:58.223606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:58.223606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:58.223606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:58.223606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:58.223606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:58.225270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:58.225270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:58.225270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:58.225270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:58.225270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:58.225270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:58.225270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:58.225270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:58.225270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:58.225270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:58.225270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:09:58.225270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:58.225270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 397/784] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:09:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:58.230795 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dacb72-f70c-4044-af6e-2e3951cb014f None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:58.304600 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65972672-642a-479c-9803-51b97829e9ba None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:58.354613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38df7b-16d2-4d42-8e59-1eb9ad1c6c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:58.355610 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38df7b-16d2-4d42-8e59-1eb9ad1c6c20 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:58.356360 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38df7b-16d2-4d42-8e59-1eb9ad1c6c20 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:58.356869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38df7b-16d2-4d42-8e59-1eb9ad1c6c20 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:58.357706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38df7b-16d2-4d42-8e59-1eb9ad1c6c20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:58.359729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38df7b-16d2-4d42-8e59-1eb9ad1c6c20 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:58.367617 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 388/785] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 7 13:09:58 2022] GET /identity/v3/services => generated 772 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:58.375171 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38df7b-16d2-4d42-8e59-1eb9ad1c6c20 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:58.469850 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dacb72-f70c-4044-af6e-2e3951cb014f None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:58.487501 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:58.512250 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:58.548289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa17a73-854a-4280-9379-4bc68d151468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:58.549990 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa17a73-854a-4280-9379-4bc68d151468 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:58.554146 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa17a73-854a-4280-9379-4bc68d151468 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:58.555581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa17a73-854a-4280-9379-4bc68d151468 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:58.560072 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa17a73-854a-4280-9379-4bc68d151468 None admin] RBAC: Adding query filter params (name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:58.561163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa17a73-854a-4280-9379-4bc68d151468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:58.564992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa17a73-854a-4280-9379-4bc68d151468 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:58.587445 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 398/786] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:09:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:58.595339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa17a73-854a-4280-9379-4bc68d151468 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:58.715080 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38df7b-16d2-4d42-8e59-1eb9ad1c6c20 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:58.811122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c719bfcf-ce93-4888-8ff1-ec7f5867dd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:58.812905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719bfcf-ce93-4888-8ff1-ec7f5867dd32 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:58.813204 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719bfcf-ce93-4888-8ff1-ec7f5867dd32 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:58.817133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719bfcf-ce93-4888-8ff1-ec7f5867dd32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:58.817364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa17a73-854a-4280-9379-4bc68d151468 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:58.849885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c719bfcf-ce93-4888-8ff1-ec7f5867dd32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:09:58.855355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-c719bfcf-ce93-4888-8ff1-ec7f5867dd32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:09:58.855355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:58.855355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:58.855355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:58.855355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:58.855355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:58.855355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:58.855355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:58.855355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:58.855355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:58.855355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:58.855355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:58.855355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:09:58.855355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:09:58.855355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:09:58.855355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:09:58.857173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:09:58.857173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:09:58.857173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:09:58.857173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:09:58.857173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:58.857173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:58.857173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:58.857173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:58.857173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:58.857173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:58.857173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:58.857173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:58.857173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:58.857173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:58.857173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:58.857173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:58.858656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:58.858656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:58.858656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:58.858656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:58.858656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:09:58.858656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:09:58.858656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:09:58.858656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:09:58.858656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:09:58.858656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:09:58.858656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:09:58.858656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:58.858656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 389/787] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:09:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 489 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:58.865727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c719bfcf-ce93-4888-8ff1-ec7f5867dd32 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:58.901645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43ddd2-dd90-4293-b3e6-97d5c3f919e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:58.902383 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43ddd2-dd90-4293-b3e6-97d5c3f919e4 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:58.902665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43ddd2-dd90-4293-b3e6-97d5c3f919e4 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:58.902967 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43ddd2-dd90-4293-b3e6-97d5c3f919e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:58.932419 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43ddd2-dd90-4293-b3e6-97d5c3f919e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:58.934229 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43ddd2-dd90-4293-b3e6-97d5c3f919e4 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:58.959063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-1c43ddd2-dd90-4293-b3e6-97d5c3f919e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:58.959063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:58.959063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:58.959063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:58.959063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:58.959063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:58.959063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:58.959063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:58.959063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:58.959063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:58.959063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:58.959063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:58.959063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:58.959063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:58.959063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:58.959063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:58.961912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:58.961912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:58.961912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:58.961912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:58.961912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:58.961912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:58.961912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:58.961912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:58.961912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:58.961912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:58.961912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:58.961912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:58.961912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:58.961912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:58.961912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:58.961912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:58.961912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:58.964607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:58.964607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:58.964607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:58.964607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 399/788] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:58.968266 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43ddd2-dd90-4293-b3e6-97d5c3f919e4 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:59.035998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:59.068696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:59.087562 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c719bfcf-ce93-4888-8ff1-ec7f5867dd32 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:59.145716 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62322ea-68cb-4b9d-a344-f33d27032bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:59.146030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43ddd2-dd90-4293-b3e6-97d5c3f919e4 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:59.147380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62322ea-68cb-4b9d-a344-f33d27032bd2 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:59.148109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62322ea-68cb-4b9d-a344-f33d27032bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:59.149298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62322ea-68cb-4b9d-a344-f33d27032bd2 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:59.151719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62322ea-68cb-4b9d-a344-f33d27032bd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:59.152046 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62322ea-68cb-4b9d-a344-f33d27032bd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:59.153466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62322ea-68cb-4b9d-a344-f33d27032bd2 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:59.172870 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 390/789] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:09:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:59.175362 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb46d9c-7cd5-45d7-a0e5-6db7cf6acd78 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:59.176265 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb46d9c-7cd5-45d7-a0e5-6db7cf6acd78 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:59.177265 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb46d9c-7cd5-45d7-a0e5-6db7cf6acd78 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:59.179845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 391/790] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:09:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:09:59.185727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb46d9c-7cd5-45d7-a0e5-6db7cf6acd78 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:59.210932 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c4d89-bb46-401d-8572-f449259b1f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:59.212030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c4d89-bb46-401d-8572-f449259b1f91 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:59.212532 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c4d89-bb46-401d-8572-f449259b1f91 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:59.212926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c4d89-bb46-401d-8572-f449259b1f91 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:59.214116 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c4d89-bb46-401d-8572-f449259b1f91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:59.214637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c4d89-bb46-401d-8572-f449259b1f91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:59.217391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c4d89-bb46-401d-8572-f449259b1f91 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:59.239219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 400/791] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:59.241173 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54b4ad-9113-4679-b2f6-7f73d99f2245 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:59.241569 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54b4ad-9113-4679-b2f6-7f73d99f2245 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:59.241940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54b4ad-9113-4679-b2f6-7f73d99f2245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:59.265918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-9a54b4ad-9113-4679-b2f6-7f73d99f2245 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:09:59.267979 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9a54b4ad-9113-4679-b2f6-7f73d99f2245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:09:59.341797 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb46d9c-7cd5-45d7-a0e5-6db7cf6acd78 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:59.376646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 401/792] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:09:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:09:59.376646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54b4ad-9113-4679-b2f6-7f73d99f2245 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:59.408781 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983398c-0f68-4a70-8b4c-8de879c8af4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:59.411302 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983398c-0f68-4a70-8b4c-8de879c8af4a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:59.413549 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983398c-0f68-4a70-8b4c-8de879c8af4a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:59.414051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983398c-0f68-4a70-8b4c-8de879c8af4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:59.439346 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983398c-0f68-4a70-8b4c-8de879c8af4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:59.442297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983398c-0f68-4a70-8b4c-8de879c8af4a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:59.449950 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:59.491731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-b983398c-0f68-4a70-8b4c-8de879c8af4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:59.491731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:59.491731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:59.491731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:59.491731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:59.491731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:59.491731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:59.491731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:59.491731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:59.491731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:59.491731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:59.491731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:59.491731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:09:59.491731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:09:59.491731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:09:59.491731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:09:59.492622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:59.492622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:59.492622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:59.492622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:59.492622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:59.492622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:59.492622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:59.492622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:59.492622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:59.492622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:59.492622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:59.492622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:59.492622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:59.492622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:59.492622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:59.492622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:59.492622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:09:59.493525 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:09:59.493525 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:09:59.493525 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:09:59.499162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 392/793] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:59.503630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983398c-0f68-4a70-8b4c-8de879c8af4a None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:59.510725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:59.670069 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54b4ad-9113-4679-b2f6-7f73d99f2245 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:59.701278 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983398c-0f68-4a70-8b4c-8de879c8af4a None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:59.768169 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e521d54-9c03-4522-91bd-939afed3068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:59.769179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e521d54-9c03-4522-91bd-939afed3068c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:59.769475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e521d54-9c03-4522-91bd-939afed3068c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:59.769771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e521d54-9c03-4522-91bd-939afed3068c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:59.775012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d75bdd-808d-4994-8908-4e0a31b69363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:59.776238 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d75bdd-808d-4994-8908-4e0a31b69363 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:59.776842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d75bdd-808d-4994-8908-4e0a31b69363 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:59.777416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d75bdd-808d-4994-8908-4e0a31b69363 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:59.779009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d75bdd-808d-4994-8908-4e0a31b69363 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:09:59.779749 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d75bdd-808d-4994-8908-4e0a31b69363 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:59.780019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e521d54-9c03-4522-91bd-939afed3068c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:59.781658 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e521d54-9c03-4522-91bd-939afed3068c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:59.783454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d75bdd-808d-4994-8908-4e0a31b69363 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:09:59.798657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-5e521d54-9c03-4522-91bd-939afed3068c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:09:59.798657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:09:59.798657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:09:59.798657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:09:59.798657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:09:59.798657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:09:59.798657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:09:59.798657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:09:59.798657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:09:59.798657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:09:59.798657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:09:59.798657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:09:59.798657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:09:59.798657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:09:59.798657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:09:59.798657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:09:59.799917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:09:59.801086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:09:59.801086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:09:59.801086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:09:59.801086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:09:59.801086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:09:59.801086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:09:59.801086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:09:59.801086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:09:59.801086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:09:59.801086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:09:59.802423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 402/794] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 7 13:09:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:09:59.802565 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e521d54-9c03-4522-91bd-939afed3068c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:59.816656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 393/795] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:09:59.818195 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d75bdd-808d-4994-8908-4e0a31b69363 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:09:59.918729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:59.932604 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:09:59.944537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e521d54-9c03-4522-91bd-939afed3068c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:59.968169 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d75bdd-808d-4994-8908-4e0a31b69363 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:09:59.992442 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efabbe-88be-4b37-8548-1bca5bd464dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:09:59.993103 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efabbe-88be-4b37-8548-1bca5bd464dd None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:09:59.993429 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efabbe-88be-4b37-8548-1bca5bd464dd None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:09:59.993701 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efabbe-88be-4b37-8548-1bca5bd464dd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:09:59.994436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16efabbe-88be-4b37-8548-1bca5bd464dd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:09:59.996273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16efabbe-88be-4b37-8548-1bca5bd464dd None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:00.009022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-16efabbe-88be-4b37-8548-1bca5bd464dd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:10:00.009022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:00.009022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:00.009022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:00.009022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:00.009022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:00.009022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:00.009022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:00.009022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:10:00.009022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:00.009022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:00.009022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:00.009022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:00.009022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:00.009022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:00.009022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:00.009022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:00.011522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:00.011522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:00.011522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:00.011522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:00.011522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:00.011522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:00.011522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:00.011522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:00.011522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:00.011522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:00.011522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:00.011522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:00.011522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:00.011522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:00.011522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:00.011522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:00.013695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:00.013695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:00.013695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:00.013695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:00.013695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:00.013695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:00.013695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:00.013695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:00.013695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:00.013695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:00.013695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:00.013695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:00.013695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 13:10:00.013695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:00.016466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb3dd2-5754-46a5-b967-fe080d65f8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:00.016466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb3dd2-5754-46a5-b967-fe080d65f8a2 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:00.016466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb3dd2-5754-46a5-b967-fe080d65f8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:00.016466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb3dd2-5754-46a5-b967-fe080d65f8a2 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:00.027634 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb3dd2-5754-46a5-b967-fe080d65f8a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:00.027634 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb3dd2-5754-46a5-b967-fe080d65f8a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:00.027634 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb3dd2-5754-46a5-b967-fe080d65f8a2 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:00.028080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 403/796] 10.222.0.62 () {56 vars in 1129 bytes} [Wed Sep 7 13:09:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:00.028080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efabbe-88be-4b37-8548-1bca5bd464dd None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:00.039565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 394/797] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Sep 7 13:09:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:00.040463 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb3dd2-5754-46a5-b967-fe080d65f8a2 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:00.159962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb3dd2-5754-46a5-b967-fe080d65f8a2 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:00.163266 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efabbe-88be-4b37-8548-1bca5bd464dd None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:00.189127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:00.204146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d964f7d-c406-4ac0-ba15-a94a54499a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:00.204948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d964f7d-c406-4ac0-ba15-a94a54499a52 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:00.205228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d964f7d-c406-4ac0-ba15-a94a54499a52 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:00.205815 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d964f7d-c406-4ac0-ba15-a94a54499a52 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:00.206473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d964f7d-c406-4ac0-ba15-a94a54499a52 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:00.206786 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d964f7d-c406-4ac0-ba15-a94a54499a52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:00.208880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d964f7d-c406-4ac0-ba15-a94a54499a52 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:00.213783 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f4964-692a-44e7-99f5-0f61d9a7f74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:00.215147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f4964-692a-44e7-99f5-0f61d9a7f74c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:00.215619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f4964-692a-44e7-99f5-0f61d9a7f74c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:00.216074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f4964-692a-44e7-99f5-0f61d9a7f74c None admin] PATH_INFO: `/v3/users/swift` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:00.219411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 395/798] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Sep 7 13:10:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:00.220772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d964f7d-c406-4ac0-ba15-a94a54499a52 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:00.229925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f4964-692a-44e7-99f5-0f61d9a7f74c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:00.232028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f4964-692a-44e7-99f5-0f61d9a7f74c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:00.248098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-635f4964-692a-44e7-99f5-0f61d9a7f74c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:10:00.248098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:00.248098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:00.248098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:00.248098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:00.248098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:00.248098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:00.248098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:00.248098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:00.248098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:00.248098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:00.248098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:00.248098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:10:00.248098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:10:00.248098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:10:00.248098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:00.249809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:00.251339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:00.251339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:00.251339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:10:00.251339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:10:00.251339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:10:00.251339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:10:00.251339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:10:00.251339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:10:00.251339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:10:00.251339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:00.254790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 404/799] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:10:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:00.256778 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f4964-692a-44e7-99f5-0f61d9a7f74c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:00.284182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:00.399378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d964f7d-c406-4ac0-ba15-a94a54499a52 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:00.445977 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f4964-692a-44e7-99f5-0f61d9a7f74c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:00.454877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e19a91-66c4-4601-a9df-9216f3514c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:00.456016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e19a91-66c4-4601-a9df-9216f3514c41 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:00.456380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e19a91-66c4-4601-a9df-9216f3514c41 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:00.456920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e19a91-66c4-4601-a9df-9216f3514c41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:00.471901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e19a91-66c4-4601-a9df-9216f3514c41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:00.473953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e19a91-66c4-4601-a9df-9216f3514c41 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:00.499818 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f14e24-6414-4669-97a2-685bc423ad33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:00.500819 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f14e24-6414-4669-97a2-685bc423ad33 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:00.501093 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f14e24-6414-4669-97a2-685bc423ad33 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:00.501367 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f14e24-6414-4669-97a2-685bc423ad33 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:00.502016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f14e24-6414-4669-97a2-685bc423ad33 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:00.502155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-06e19a91-66c4-4601-a9df-9216f3514c41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:00.502155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:00.502155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:00.502155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:00.502155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:00.502155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:00.502155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:00.502155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:00.502155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:00.502155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:00.502155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:00.502155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:00.502155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:00.502155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:00.502155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:00.502155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:00.503863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:00.503863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:00.503863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:00.503863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:00.503863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:00.503863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:00.503863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:00.503863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:00.503863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:00.503863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:00.503863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:00.503863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:00.503863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:00.503863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:00.503863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:00.503863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:00.503863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:00.504950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:00.504950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:00.504950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:00.505165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f14e24-6414-4669-97a2-685bc423ad33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:00.505165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f14e24-6414-4669-97a2-685bc423ad33 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:00.507201 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 396/800] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:00.508481 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e19a91-66c4-4601-a9df-9216f3514c41 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:00.518243 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 405/801] 10.222.0.62 () {56 vars in 1124 bytes} [Wed Sep 7 13:10:00 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:00.519953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f14e24-6414-4669-97a2-685bc423ad33 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:00.581705 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:00.635023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:00.662854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f14e24-6414-4669-97a2-685bc423ad33 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:00.683859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e19a91-66c4-4601-a9df-9216f3514c41 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:00.708122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57626aa3-064c-4b2e-b05a-69de466ecad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:00.709121 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57626aa3-064c-4b2e-b05a-69de466ecad8 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:00.709654 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57626aa3-064c-4b2e-b05a-69de466ecad8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:00.710128 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57626aa3-064c-4b2e-b05a-69de466ecad8 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:00.711060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57626aa3-064c-4b2e-b05a-69de466ecad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:00.711736 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57626aa3-064c-4b2e-b05a-69de466ecad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:00.714181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57626aa3-064c-4b2e-b05a-69de466ecad8 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:00.732403 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 406/802] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:00.732635 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0fe696-0b20-4e4b-ae83-b43957be1232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:00.733390 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fe696-0b20-4e4b-ae83-b43957be1232 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:00.734616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57626aa3-064c-4b2e-b05a-69de466ecad8 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:00.741412 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fe696-0b20-4e4b-ae83-b43957be1232 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:00.742112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fe696-0b20-4e4b-ae83-b43957be1232 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:00.743598 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0fe696-0b20-4e4b-ae83-b43957be1232 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:00.744143 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0fe696-0b20-4e4b-ae83-b43957be1232 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:00.746936 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0fe696-0b20-4e4b-ae83-b43957be1232 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:00.790128 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 397/803] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Sep 7 13:10:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:00.790128 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0fe696-0b20-4e4b-ae83-b43957be1232 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:00.912051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57626aa3-064c-4b2e-b05a-69de466ecad8 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:00.962270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:00.971852 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0fe696-0b20-4e4b-ae83-b43957be1232 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:00.973272 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531da51e-0c36-4817-9018-547e22daa508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:00.976502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531da51e-0c36-4817-9018-547e22daa508 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:00.976502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531da51e-0c36-4817-9018-547e22daa508 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:00.976502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531da51e-0c36-4817-9018-547e22daa508 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:00.976502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531da51e-0c36-4817-9018-547e22daa508 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:00.978215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531da51e-0c36-4817-9018-547e22daa508 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:00.992855 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:01.018780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395b434-9fea-4216-9098-2b176e3350c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:01.019293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395b434-9fea-4216-9098-2b176e3350c3 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:01.019397 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395b434-9fea-4216-9098-2b176e3350c3 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:01.019653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395b434-9fea-4216-9098-2b176e3350c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:01.029872 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d395b434-9fea-4216-9098-2b176e3350c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:01.032088 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d395b434-9fea-4216-9098-2b176e3350c3 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:01.045703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-d395b434-9fea-4216-9098-2b176e3350c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:01.045703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:01.045703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:01.045703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:01.045703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:01.045703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:01.045703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:01.045703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:01.045703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:01.045703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:01.045703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:01.045703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:01.045703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:10:01.045703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:10:01.045703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:10:01.048806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:10:01.048806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:01.048806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:01.048806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:01.048806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:01.048806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:01.048806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:01.048806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:01.048806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:01.048806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:01.048806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:01.048806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:01.048806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:01.048806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:01.048806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:01.048806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:01.048806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:01.050606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:10:01.050606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:10:01.050606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:10:01.050606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:10:01.050606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:10:01.050606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:10:01.050606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:01.050606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:01.053282 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 398/804] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:10:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:01.055658 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395b434-9fea-4216-9098-2b176e3350c3 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:01.097701 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 407/805] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:10:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:10:01.097701 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531da51e-0c36-4817-9018-547e22daa508 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:01.297950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395b434-9fea-4216-9098-2b176e3350c3 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:01.304258 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531da51e-0c36-4817-9018-547e22daa508 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:01.369016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86fab3-294c-4d7c-a528-fb617a012931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:01.370310 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86fab3-294c-4d7c-a528-fb617a012931 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:01.370785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86fab3-294c-4d7c-a528-fb617a012931 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:01.371276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86fab3-294c-4d7c-a528-fb617a012931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:01.377265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eba7ed-dae3-4f09-a268-e6d392f93258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:01.378837 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eba7ed-dae3-4f09-a268-e6d392f93258 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:01.379676 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eba7ed-dae3-4f09-a268-e6d392f93258 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:01.380426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eba7ed-dae3-4f09-a268-e6d392f93258 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:01.382179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eba7ed-dae3-4f09-a268-e6d392f93258 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:01.382798 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eba7ed-dae3-4f09-a268-e6d392f93258 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:01.385914 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eba7ed-dae3-4f09-a268-e6d392f93258 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:01.394809 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86fab3-294c-4d7c-a528-fb617a012931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:01.397460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86fab3-294c-4d7c-a528-fb617a012931 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:01.414528 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 408/806] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:10:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:01.423028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53d461-18e8-4f84-858c-9101bf3481fe None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:01.424234 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53d461-18e8-4f84-858c-9101bf3481fe None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:01.424918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53d461-18e8-4f84-858c-9101bf3481fe None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:01.427062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-ec86fab3-294c-4d7c-a528-fb617a012931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:01.427062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:01.427062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:01.427062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:01.427062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:01.427062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:01.427062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:01.427062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:01.427062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:01.427062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:01.427062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:01.427062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:01.427062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:01.427062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:01.427062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:01.427062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:01.428822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:01.428822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:01.428822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:01.428822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:01.428822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:01.428822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:01.428822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:01.428822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:01.428822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:01.428822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:01.428822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:01.428822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:01.428822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:01.428822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:01.428822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:01.428822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:01.428822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:01.430521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:01.430521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:01.430521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:01.430521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 399/807] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:01.433394 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 409/808] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:01 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:01.440253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86fab3-294c-4d7c-a528-fb617a012931 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:01.442068 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53d461-18e8-4f84-858c-9101bf3481fe None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:01.461868 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:01.543759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:01.621587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86fab3-294c-4d7c-a528-fb617a012931 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:01.643982 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53d461-18e8-4f84-858c-9101bf3481fe None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:01.672102 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30e93e-c195-49ee-9305-1af496657547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:01.672983 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30e93e-c195-49ee-9305-1af496657547 None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:01.673258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30e93e-c195-49ee-9305-1af496657547 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:01.673541 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30e93e-c195-49ee-9305-1af496657547 None admin] PATH_INFO: `/v3/projects/4574cf3ff270485faf60594019f7e05c/users/8121ba4cf2cf44d9a5a55975ea85928c/roles/743ed2d8e7114975bd9245742a41fe0f` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:01.698071 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30e93e-c195-49ee-9305-1af496657547 None admin] RBAC: Authorizing `identity:create_grant(project_id=4574cf3ff270485faf60594019f7e05c, user_id=8121ba4cf2cf44d9a5a55975ea85928c, role_id=743ed2d8e7114975bd9245742a41fe0f)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:01.700738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30e93e-c195-49ee-9305-1af496657547 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:01.716206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac710e1-65b0-4254-958b-99707b6773aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:01.716206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac710e1-65b0-4254-958b-99707b6773aa None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:01.716509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac710e1-65b0-4254-958b-99707b6773aa None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:01.716606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac710e1-65b0-4254-958b-99707b6773aa None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:01.717552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac710e1-65b0-4254-958b-99707b6773aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:01.717717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac710e1-65b0-4254-958b-99707b6773aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:01.720512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac710e1-65b0-4254-958b-99707b6773aa None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:01.739657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 410/809] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:01.741021 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dcee90-838c-4b2b-82fc-a410cc24e60b None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:01.741312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dcee90-838c-4b2b-82fc-a410cc24e60b None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:01.741589 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dcee90-838c-4b2b-82fc-a410cc24e60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:01.769874 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-d5dcee90-838c-4b2b-82fc-a410cc24e60b None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:01.771438 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d5dcee90-838c-4b2b-82fc-a410cc24e60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:01.817483 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 400/810] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:10:01 2022] PUT /identity/v3/projects/4574cf3ff270485faf60594019f7e05c/users/8121ba4cf2cf44d9a5a55975ea85928c/roles/743ed2d8e7114975bd9245742a41fe0f => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:10:01.819962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd5869-56ff-4d1f-a577-20f04c156fa1 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:01.820466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd5869-56ff-4d1f-a577-20f04c156fa1 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:01.821147 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd5869-56ff-4d1f-a577-20f04c156fa1 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:01.828503 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 401/811] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:01 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:01.830667 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd5869-56ff-4d1f-a577-20f04c156fa1 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:01.911453 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 411/812] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:01.913272 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff948dc5-27da-464e-a345-a2daab69b65d None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:01.913864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff948dc5-27da-464e-a345-a2daab69b65d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:01.914350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff948dc5-27da-464e-a345-a2daab69b65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:01.939560 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-ff948dc5-27da-464e-a345-a2daab69b65d None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:01.941343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ff948dc5-27da-464e-a345-a2daab69b65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:01.977295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:02.006948 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:02.060692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 412/813] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:02.100339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd5869-56ff-4d1f-a577-20f04c156fa1 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:02.158794 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34421914-46ed-4771-b13d-f06fb2ae0a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:02.160260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34421914-46ed-4771-b13d-f06fb2ae0a54 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:02.160957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34421914-46ed-4771-b13d-f06fb2ae0a54 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:02.161698 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34421914-46ed-4771-b13d-f06fb2ae0a54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:02.173628 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34421914-46ed-4771-b13d-f06fb2ae0a54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:02.177137 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34421914-46ed-4771-b13d-f06fb2ae0a54 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:02.198467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-34421914-46ed-4771-b13d-f06fb2ae0a54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:02.198467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:02.198467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:02.198467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:02.198467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:02.198467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:02.198467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:02.198467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:02.198467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:02.198467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:02.198467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:02.198467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:02.198467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:10:02.198467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:10:02.198467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:10:02.199929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:10:02.199929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:02.199929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:02.199929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:02.199929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:02.199929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:02.199929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:02.199929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:02.199929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:02.199929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:02.199929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:02.199929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:02.199929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:02.199929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:02.199929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:02.199929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:02.199929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:02.201844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:10:02.201844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:10:02.201844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:10:02.201844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:10:02.201844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:10:02.201844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:10:02.201844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:02.201844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:02.207599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 402/814] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:10:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:02.217335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff948dc5-27da-464e-a345-a2daab69b65d None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:02.372930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff948dc5-27da-464e-a345-a2daab69b65d None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:02.395284 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34421914-46ed-4771-b13d-f06fb2ae0a54 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:02.435565 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e29702-beb0-4b76-a8fa-7a95f53ca83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:02.438616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e29702-beb0-4b76-a8fa-7a95f53ca83e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:02.439267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e29702-beb0-4b76-a8fa-7a95f53ca83e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:02.439767 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e29702-beb0-4b76-a8fa-7a95f53ca83e None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:02.441245 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e29702-beb0-4b76-a8fa-7a95f53ca83e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:02.441829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e29702-beb0-4b76-a8fa-7a95f53ca83e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:02.443970 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e29702-beb0-4b76-a8fa-7a95f53ca83e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:02.464622 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 413/815] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:10:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:02.473431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e29702-beb0-4b76-a8fa-7a95f53ca83e None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:02.603576 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:02.682609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34421914-46ed-4771-b13d-f06fb2ae0a54 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:02.719585 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:02.755921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fb0e2-f8d8-4b6c-8835-0f60719e524c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:02.756629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fb0e2-f8d8-4b6c-8835-0f60719e524c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:02.756914 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fb0e2-f8d8-4b6c-8835-0f60719e524c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:02.757186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fb0e2-f8d8-4b6c-8835-0f60719e524c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:02.758369 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e29702-beb0-4b76-a8fa-7a95f53ca83e None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:02.773509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fb0e2-f8d8-4b6c-8835-0f60719e524c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:10:02.779215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-8d2fb0e2-f8d8-4b6c-8835-0f60719e524c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:10:02.779215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:02.779215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:02.779215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:02.779215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:02.779215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:02.779215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:02.779215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:02.779215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:02.779215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:02.779215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:02.779215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:02.779215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:10:02.779215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:10:02.779215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:10:02.779215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:10:02.781675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:10:02.781675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:10:02.781675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:10:02.781675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:10:02.781675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:02.781675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:02.781675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:02.781675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:02.781675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:02.781675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:02.781675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:02.781675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:02.781675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:02.781675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:02.781675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:02.781675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:02.783228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:02.783228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:02.783228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:02.783228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:02.783228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:10:02.783228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:10:02.783228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:10:02.783228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:10:02.783228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:10:02.783228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:10:02.783228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:10:02.783228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:02.786687 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 403/816] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:10:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:02.788064 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fb0e2-f8d8-4b6c-8835-0f60719e524c None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:02.852084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4088f-6683-44ca-b9b2-c5e74a9d69d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:02.853251 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4088f-6683-44ca-b9b2-c5e74a9d69d2 None admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:02.853755 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4088f-6683-44ca-b9b2-c5e74a9d69d2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:02.854255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4088f-6683-44ca-b9b2-c5e74a9d69d2 None admin] PATH_INFO: `/v3/projects/4574cf3ff270485faf60594019f7e05c/users/5cedc7091d2f46b5be68818d26399e9b/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:02.858179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4088f-6683-44ca-b9b2-c5e74a9d69d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4574cf3ff270485faf60594019f7e05c, user_id=5cedc7091d2f46b5be68818d26399e9b, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:02.861019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4088f-6683-44ca-b9b2-c5e74a9d69d2 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:02.996358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 414/817] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:10:02 2022] PUT /identity/v3/projects/4574cf3ff270485faf60594019f7e05c/users/5cedc7091d2f46b5be68818d26399e9b/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:10:02.996358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4088f-6683-44ca-b9b2-c5e74a9d69d2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:02.996717 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fb0e2-f8d8-4b6c-8835-0f60719e524c None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:03.132634 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f3607-af1d-4a70-a580-0ba8f90b22f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:03.133433 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f3607-af1d-4a70-a580-0ba8f90b22f8 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:03.133692 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f3607-af1d-4a70-a580-0ba8f90b22f8 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:03.133998 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f3607-af1d-4a70-a580-0ba8f90b22f8 None admin] PATH_INFO: `/v3/services/network` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:03.134623 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f3607-af1d-4a70-a580-0ba8f90b22f8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:03.136332 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f3607-af1d-4a70-a580-0ba8f90b22f8 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:03.139635 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:03.152122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4088f-6683-44ca-b9b2-c5e74a9d69d2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:03.177941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-086f3607-af1d-4a70-a580-0ba8f90b22f8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:10:03.177941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:03.177941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:03.177941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:03.177941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:03.177941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:03.177941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:03.177941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:03.177941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:10:03.177941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:03.177941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:03.177941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:03.177941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:03.177941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:03.177941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:03.177941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:03.177941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:03.179174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:03.179174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:03.179174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:03.179174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:03.179174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:03.179174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:03.179174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:03.179174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:03.179174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:03.179174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:03.179174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:03.179174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:03.179174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:03.179174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:03.179174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:03.179174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:03.180059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:03.180059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:03.180059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:03.180059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:03.180059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:03.180059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:03.180059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:03.180059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:03.180059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:03.180059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:03.180059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:03.180059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:03.180059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 13:10:03.180059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:03.180059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 404/818] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:10:02 2022] GET /identity/v3/services/network => generated 88 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:03.186369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f3607-af1d-4a70-a580-0ba8f90b22f8 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:03.207456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f74be-57cd-42a4-836c-1064acb92b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:03.209961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f74be-57cd-42a4-836c-1064acb92b09 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:03.210591 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f74be-57cd-42a4-836c-1064acb92b09 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:03.211114 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f74be-57cd-42a4-836c-1064acb92b09 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:03.212264 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f74be-57cd-42a4-836c-1064acb92b09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:03.212910 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f74be-57cd-42a4-836c-1064acb92b09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:03.214767 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f74be-57cd-42a4-836c-1064acb92b09 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:03.228117 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 415/819] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:10:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:03.238103 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f74be-57cd-42a4-836c-1064acb92b09 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:03.397423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f74be-57cd-42a4-836c-1064acb92b09 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:03.406677 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:03.447657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aee408-5a9a-4e34-9822-4b5cb4f20acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:03.448677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aee408-5a9a-4e34-9822-4b5cb4f20acd None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:03.449102 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aee408-5a9a-4e34-9822-4b5cb4f20acd None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:03.449599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aee408-5a9a-4e34-9822-4b5cb4f20acd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:03.464601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aee408-5a9a-4e34-9822-4b5cb4f20acd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:03.466601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aee408-5a9a-4e34-9822-4b5cb4f20acd None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:03.489431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-e5aee408-5a9a-4e34-9822-4b5cb4f20acd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:03.489431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:03.489431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:03.489431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:03.489431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:03.489431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:03.489431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:03.489431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:03.489431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:03.489431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:03.489431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:03.489431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:03.489431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:03.489431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:03.489431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:03.489431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:03.490545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:03.490545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:03.490545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:03.490545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:03.490545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:03.490545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:03.490545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:03.490545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:03.490545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:03.490545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:03.490545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:03.490545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:03.490545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:03.490545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:03.490545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:03.490545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:03.490545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:03.492343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:03.492343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:03.492343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:03.492343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 416/820] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:03.501974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aee408-5a9a-4e34-9822-4b5cb4f20acd None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:03.523225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:03.531784 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f3607-af1d-4a70-a580-0ba8f90b22f8 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:03.578711 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6bcdc-b06a-4b87-9aea-86ba71efecc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:03.580176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6bcdc-b06a-4b87-9aea-86ba71efecc4 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:03.581070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6bcdc-b06a-4b87-9aea-86ba71efecc4 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:03.581622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6bcdc-b06a-4b87-9aea-86ba71efecc4 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:03.582552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6bcdc-b06a-4b87-9aea-86ba71efecc4 None admin] RBAC: Adding query filter params (name=network) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:03.583264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6bcdc-b06a-4b87-9aea-86ba71efecc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:03.585111 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6bcdc-b06a-4b87-9aea-86ba71efecc4 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:03.598475 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 405/821] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 7 13:10:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:03.600315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6bcdc-b06a-4b87-9aea-86ba71efecc4 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:03.726244 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aee408-5a9a-4e34-9822-4b5cb4f20acd None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:03.763558 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6bcdc-b06a-4b87-9aea-86ba71efecc4 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:03.779499 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:03.835536 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa5139-fa59-48b0-a062-7707ccd3c68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:03.837106 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa5139-fa59-48b0-a062-7707ccd3c68a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:03.837878 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa5139-fa59-48b0-a062-7707ccd3c68a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:03.838643 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa5139-fa59-48b0-a062-7707ccd3c68a None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:03.840147 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa5139-fa59-48b0-a062-7707ccd3c68a None admin] RBAC: Adding query filter params (type=network) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:03.841031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa5139-fa59-48b0-a062-7707ccd3c68a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:03.844091 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa5139-fa59-48b0-a062-7707ccd3c68a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:03.859479 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 406/822] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 7 13:10:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:03.864627 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7d747-0447-46b1-b0ac-2041fb8913dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:03.865689 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7d747-0447-46b1-b0ac-2041fb8913dd None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:03.866172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7d747-0447-46b1-b0ac-2041fb8913dd None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:03.866639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7d747-0447-46b1-b0ac-2041fb8913dd None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:03.867280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa5139-fa59-48b0-a062-7707ccd3c68a None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:03.867936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7d747-0447-46b1-b0ac-2041fb8913dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:03.868505 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7d747-0447-46b1-b0ac-2041fb8913dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:03.871140 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7d747-0447-46b1-b0ac-2041fb8913dd None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:03.902047 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 417/823] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:03.911715 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7d747-0447-46b1-b0ac-2041fb8913dd None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:04.074091 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa5139-fa59-48b0-a062-7707ccd3c68a None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:04.121380 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:04.144972 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3e90d-22ca-43d6-8b29-405e21d54aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:04.146546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3e90d-22ca-43d6-8b29-405e21d54aeb None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:04.147361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3e90d-22ca-43d6-8b29-405e21d54aeb None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:04.148157 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3e90d-22ca-43d6-8b29-405e21d54aeb None admin] PATH_INFO: `/v3/endpoints` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:04.149711 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3e90d-22ca-43d6-8b29-405e21d54aeb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:04.152662 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3e90d-22ca-43d6-8b29-405e21d54aeb None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:04.198532 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7d747-0447-46b1-b0ac-2041fb8913dd None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:04.256661 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef88a4-4c38-4e5e-87dd-85bd0d491b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:04.257447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef88a4-4c38-4e5e-87dd-85bd0d491b9a None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:04.257798 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef88a4-4c38-4e5e-87dd-85bd0d491b9a None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:04.258100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef88a4-4c38-4e5e-87dd-85bd0d491b9a None admin] PATH_INFO: `/v3/users/nova` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:04.269417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef88a4-4c38-4e5e-87dd-85bd0d491b9a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:04.270885 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef88a4-4c38-4e5e-87dd-85bd0d491b9a None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:04.285051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-69ef88a4-4c38-4e5e-87dd-85bd0d491b9a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:10:04.285051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:04.285051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:04.285051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:04.285051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:04.285051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:04.285051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:04.285051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:04.285051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:04.285051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:04.285051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:04.285051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:04.285051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:10:04.285051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:10:04.285051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:10:04.285051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:04.294412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:04.296488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:04.296488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:04.296488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:10:04.296488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:10:04.296488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:10:04.296488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:10:04.296488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:10:04.296488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:10:04.296488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:10:04.296488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:04.296488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 418/824] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 7 13:10:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:04.296488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef88a4-4c38-4e5e-87dd-85bd0d491b9a None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:04.304025 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 407/825] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:10:03 2022] POST /identity/v3/endpoints => generated 337 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:10:04.542416 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef88a4-4c38-4e5e-87dd-85bd0d491b9a None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:04.614048 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:04.615144 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e02bd9-578b-4e5a-8fe9-b9c6a0af380a None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:04.615447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e02bd9-578b-4e5a-8fe9-b9c6a0af380a None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:04.615778 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e02bd9-578b-4e5a-8fe9-b9c6a0af380a None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:04.618257 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 408/826] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:10:04.633339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ddc84-bce1-4b8e-acbe-21802d7d58f1 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:04.634240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ddc84-bce1-4b8e-acbe-21802d7d58f1 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:04.634240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ddc84-bce1-4b8e-acbe-21802d7d58f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:04.642336 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20994ffe-ce70-478a-be1b-ab38e564e548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:04.643547 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20994ffe-ce70-478a-be1b-ab38e564e548 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:04.644164 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20994ffe-ce70-478a-be1b-ab38e564e548 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:04.644711 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20994ffe-ce70-478a-be1b-ab38e564e548 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:04.645920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20994ffe-ce70-478a-be1b-ab38e564e548 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:04.646444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20994ffe-ce70-478a-be1b-ab38e564e548 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:04.649105 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20994ffe-ce70-478a-be1b-ab38e564e548 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:04.677741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-b85ddc84-bce1-4b8e-acbe-21802d7d58f1 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:04.679713 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b85ddc84-bce1-4b8e-acbe-21802d7d58f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:04.695080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 419/827] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 7 13:10:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:04.696684 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20994ffe-ce70-478a-be1b-ab38e564e548 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:04.712310 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:04.822534 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 409/828] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Sep 07 13:10:04.891885 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20994ffe-ce70-478a-be1b-ab38e564e548 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:04.946157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743a462-e952-4599-93cd-db9932e93ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:04.947108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743a462-e952-4599-93cd-db9932e93ca5 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:04.947518 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743a462-e952-4599-93cd-db9932e93ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:04.947903 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743a462-e952-4599-93cd-db9932e93ca5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:05.049530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:05.086589 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743a462-e952-4599-93cd-db9932e93ca5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:05.088187 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743a462-e952-4599-93cd-db9932e93ca5 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:05.242682 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 420/829] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:10:04 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 13:10:05.251437 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ddc84-bce1-4b8e-acbe-21802d7d58f1 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:05.252203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743a462-e952-4599-93cd-db9932e93ca5 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:05.361066 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:05.396474 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743a462-e952-4599-93cd-db9932e93ca5 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:05.429848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ddc84-bce1-4b8e-acbe-21802d7d58f1 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:05.443258 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b57c5-a693-4063-a383-ea3e2099f5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:05.443970 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b57c5-a693-4063-a383-ea3e2099f5f6 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:05.444254 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b57c5-a693-4063-a383-ea3e2099f5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:05.444660 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b57c5-a693-4063-a383-ea3e2099f5f6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:05.445355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b57c5-a693-4063-a383-ea3e2099f5f6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:05.446846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b57c5-a693-4063-a383-ea3e2099f5f6 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:05.459204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-cd2b57c5-a693-4063-a383-ea3e2099f5f6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:10:05.459204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:05.459204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:05.459204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:05.459204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:05.459204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:05.459204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:05.459204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:05.459204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:10:05.459204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:05.459204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:05.459204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:05.459204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:05.459204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:05.459204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:05.459204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:05.459204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:05.462073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:05.462073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:05.462073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:05.462073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:05.462073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:05.462073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:05.462073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:05.462073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:05.462073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:05.462073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:05.462073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:05.462073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:05.462073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:05.462073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:05.462073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:05.462073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:05.464858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:05.464858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:05.464858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:05.464858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:05.464858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:05.464858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:05.464858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:05.464858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:05.464858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:05.464858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:05.464858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:05.464858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:05.464858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:10:05.464858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:05.464858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 421/830] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:10:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:05.474433 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b57c5-a693-4063-a383-ea3e2099f5f6 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:05.510664 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad805ab-9a95-4443-b0af-ad79da2120ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:05.512186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad805ab-9a95-4443-b0af-ad79da2120ab None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:05.512952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad805ab-9a95-4443-b0af-ad79da2120ab None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:05.513684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad805ab-9a95-4443-b0af-ad79da2120ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:05.534511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad805ab-9a95-4443-b0af-ad79da2120ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:05.537653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad805ab-9a95-4443-b0af-ad79da2120ab None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:05.569016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-7ad805ab-9a95-4443-b0af-ad79da2120ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:05.569016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:05.569016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:05.569016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:05.569016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:05.569016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:05.569016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:05.569016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:05.569016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:05.569016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:05.569016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:05.569016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:05.569016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:05.569016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:05.569016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:05.569016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:05.570899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:05.570899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:05.570899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:05.570899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:05.570899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:05.570899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:05.570899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:05.570899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:05.570899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:05.570899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:05.570899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:05.570899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:05.570899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:05.570899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:05.570899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:05.570899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:05.570899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:05.573129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:05.573129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:05.573129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:05.576496 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 410/831] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:05.586482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad805ab-9a95-4443-b0af-ad79da2120ab None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:05.612348 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:05.695706 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b57c5-a693-4063-a383-ea3e2099f5f6 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:05.780598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64beb297-bba4-4b05-acca-e9b4824039c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:05.781279 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64beb297-bba4-4b05-acca-e9b4824039c2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:05.781777 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64beb297-bba4-4b05-acca-e9b4824039c2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:05.782269 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64beb297-bba4-4b05-acca-e9b4824039c2 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:05.783512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64beb297-bba4-4b05-acca-e9b4824039c2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:05.784228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64beb297-bba4-4b05-acca-e9b4824039c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:05.786627 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64beb297-bba4-4b05-acca-e9b4824039c2 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:05.795487 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 422/832] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:10:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:05.805585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64beb297-bba4-4b05-acca-e9b4824039c2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:05.821136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad805ab-9a95-4443-b0af-ad79da2120ab None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:05.850218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:05.902471 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4d835-77fb-4b97-ae98-a311ed771613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:05.904070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4d835-77fb-4b97-ae98-a311ed771613 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:05.905274 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4d835-77fb-4b97-ae98-a311ed771613 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:05.907399 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4d835-77fb-4b97-ae98-a311ed771613 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:05.908965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4d835-77fb-4b97-ae98-a311ed771613 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:05.909822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4d835-77fb-4b97-ae98-a311ed771613 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:05.913507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4d835-77fb-4b97-ae98-a311ed771613 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:05.938056 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 411/833] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:05.947695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4d835-77fb-4b97-ae98-a311ed771613 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:06.021317 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64beb297-bba4-4b05-acca-e9b4824039c2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:06.087650 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:06.095741 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008606c4-22fb-4777-bc30-8bb56a9839d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:06.097295 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008606c4-22fb-4777-bc30-8bb56a9839d9 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:06.098041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008606c4-22fb-4777-bc30-8bb56a9839d9 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:06.098729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008606c4-22fb-4777-bc30-8bb56a9839d9 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:06.100204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008606c4-22fb-4777-bc30-8bb56a9839d9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:06.103101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008606c4-22fb-4777-bc30-8bb56a9839d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:06.104061 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008606c4-22fb-4777-bc30-8bb56a9839d9 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:06.114694 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 423/834] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:10:05 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:06.117426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95d78b-88fb-41ea-8b3b-e36b1ba41031 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:06.118256 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95d78b-88fb-41ea-8b3b-e36b1ba41031 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:06.118965 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95d78b-88fb-41ea-8b3b-e36b1ba41031 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:06.122661 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 424/835] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:06.136208 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47407b69-80de-4d62-b71c-14a6f8fea8b9 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:06.137193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47407b69-80de-4d62-b71c-14a6f8fea8b9 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:06.138060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47407b69-80de-4d62-b71c-14a6f8fea8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:06.147592 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4d835-77fb-4b97-ae98-a311ed771613 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:06.170829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-47407b69-80de-4d62-b71c-14a6f8fea8b9 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:06.173510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-47407b69-80de-4d62-b71c-14a6f8fea8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:06.200945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc18da-3983-447b-9e7a-5586f96e24d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:06.201620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc18da-3983-447b-9e7a-5586f96e24d5 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:06.201935 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc18da-3983-447b-9e7a-5586f96e24d5 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:06.202212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc18da-3983-447b-9e7a-5586f96e24d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:06.213525 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc18da-3983-447b-9e7a-5586f96e24d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:06.215612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc18da-3983-447b-9e7a-5586f96e24d5 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:06.233072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-21fc18da-3983-447b-9e7a-5586f96e24d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:06.233072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:06.233072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:06.233072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:06.233072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:06.233072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:06.233072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:06.233072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:06.233072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:06.233072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:06.233072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:06.233072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:06.233072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:10:06.233072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:10:06.233072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:10:06.234804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:10:06.234804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:06.234804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:06.234804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:06.234804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:06.234804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:06.234804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:06.234804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:06.234804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:06.234804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:06.234804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:06.234804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:06.234804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:06.234804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:06.234804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:06.234804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:06.234804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:06.237229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:10:06.237229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:10:06.237229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:10:06.237229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:10:06.237229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:10:06.237229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:10:06.237229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:06.237229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:06.239993 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 412/836] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:10:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:06.242935 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc18da-3983-447b-9e7a-5586f96e24d5 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:06.319259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 425/837] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:10:06.469931 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc18da-3983-447b-9e7a-5586f96e24d5 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:06.532361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:06.540807 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba2b01-36db-4833-9cf8-498d3c3ed5a2 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:06.541190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba2b01-36db-4833-9cf8-498d3c3ed5a2 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:06.541379 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba2b01-36db-4833-9cf8-498d3c3ed5a2 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:06.545292 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1fa26-0450-4a86-aa88-18e1f7c6f8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:06.546376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1fa26-0450-4a86-aa88-18e1f7c6f8e2 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:06.552178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1fa26-0450-4a86-aa88-18e1f7c6f8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:06.552178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1fa26-0450-4a86-aa88-18e1f7c6f8e2 None admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:06.552530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 426/838] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:06.552668 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1fa26-0450-4a86-aa88-18e1f7c6f8e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:06.552668 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1fa26-0450-4a86-aa88-18e1f7c6f8e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:06.552911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e9423-9a52-4821-a8bc-6a0347eff196 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:06.552911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e9423-9a52-4821-a8bc-6a0347eff196 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:06.552911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e9423-9a52-4821-a8bc-6a0347eff196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:06.553284 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1fa26-0450-4a86-aa88-18e1f7c6f8e2 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:06.571584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:06.579455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-124e9423-9a52-4821-a8bc-6a0347eff196 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:06.580684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 413/839] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:10:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:06.581731 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-124e9423-9a52-4821-a8bc-6a0347eff196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:06.590612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1fa26-0450-4a86-aa88-18e1f7c6f8e2 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:06.732697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 427/840] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:10:06.809501 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1fa26-0450-4a86-aa88-18e1f7c6f8e2 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:06.835308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e9423-9a52-4821-a8bc-6a0347eff196 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:06.877308 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404d7bd-da60-40ba-b3fd-c7524f16da67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:06.878345 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404d7bd-da60-40ba-b3fd-c7524f16da67 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:06.878906 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404d7bd-da60-40ba-b3fd-c7524f16da67 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:06.879432 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404d7bd-da60-40ba-b3fd-c7524f16da67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:06.880771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404d7bd-da60-40ba-b3fd-c7524f16da67 None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=97c363046db444cbb3d4346066f9ad42) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:06.881319 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404d7bd-da60-40ba-b3fd-c7524f16da67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:06.882842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404d7bd-da60-40ba-b3fd-c7524f16da67 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:06.891386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 414/841] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:10:06 2022] GET /identity/v3/role_assignments?user.id=97c363046db444cbb3d4346066f9ad42&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:07.038160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e9423-9a52-4821-a8bc-6a0347eff196 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:07.087193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:07.094360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ab90f-d226-4638-9fef-d5c2d92dcb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:07.100456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ab90f-d226-4638-9fef-d5c2d92dcb2f None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:07.100456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ab90f-d226-4638-9fef-d5c2d92dcb2f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:07.100456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ab90f-d226-4638-9fef-d5c2d92dcb2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:07.115272 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ab90f-d226-4638-9fef-d5c2d92dcb2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:10:07.123959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-5a6ab90f-d226-4638-9fef-d5c2d92dcb2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:10:07.123959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:07.123959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:07.123959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:07.123959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:07.123959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:07.123959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:07.123959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:07.123959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:07.123959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:07.123959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:07.123959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:07.123959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:10:07.123959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:10:07.123959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:10:07.123959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:10:07.126202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:10:07.126202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:10:07.126202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:10:07.126202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:10:07.126202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:07.126202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:07.126202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:07.126202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:07.126202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:07.126202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:07.126202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:07.126202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:07.126202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:07.126202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:07.126202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:07.126202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:07.127790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:07.127790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:07.127790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:07.127790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:07.127790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:10:07.127790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:10:07.127790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:10:07.127790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:10:07.127790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:10:07.127790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:10:07.127790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:10:07.127790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:07.134635 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 428/842] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:10:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:07.136688 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404d7bd-da60-40ba-b3fd-c7524f16da67 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:07.170377 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ab90f-d226-4638-9fef-d5c2d92dcb2f None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:07.262767 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:07.370074 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404d7bd-da60-40ba-b3fd-c7524f16da67 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:07.391955 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ab90f-d226-4638-9fef-d5c2d92dcb2f None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:07.424780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6715b-8be7-47bb-aceb-aff3f85f7447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:07.425848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6715b-8be7-47bb-aceb-aff3f85f7447 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:07.426336 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6715b-8be7-47bb-aceb-aff3f85f7447 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:07.426902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6715b-8be7-47bb-aceb-aff3f85f7447 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:07.428323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6715b-8be7-47bb-aceb-aff3f85f7447 None admin] RBAC: Adding query filter params (name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:07.428859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6715b-8be7-47bb-aceb-aff3f85f7447 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:07.431755 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6715b-8be7-47bb-aceb-aff3f85f7447 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:07.448085 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 415/843] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:10:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:07.458454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6715b-8be7-47bb-aceb-aff3f85f7447 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:07.460154 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f69a35-bfc1-4190-8b3e-fdcb602add06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:07.460947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f69a35-bfc1-4190-8b3e-fdcb602add06 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:07.461290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f69a35-bfc1-4190-8b3e-fdcb602add06 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:07.461560 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f69a35-bfc1-4190-8b3e-fdcb602add06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:07.477922 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f69a35-bfc1-4190-8b3e-fdcb602add06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:10:07.485448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-f3f69a35-bfc1-4190-8b3e-fdcb602add06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:10:07.485448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:07.485448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:07.485448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:07.485448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:07.485448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:07.485448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:07.485448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:07.485448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:07.485448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:07.485448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:07.485448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:07.485448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:10:07.485448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:10:07.485448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:10:07.485448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:10:07.487388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:10:07.487388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:10:07.487388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:10:07.487388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:10:07.487388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:07.487388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:07.487388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:07.487388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:07.487388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:07.487388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:07.487388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:07.487388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:07.487388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:07.487388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:07.487388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:07.487388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:07.489041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:07.489041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:07.489041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:07.489041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:07.489041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:10:07.489041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:10:07.489041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:10:07.489041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:10:07.489041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:10:07.489041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:10:07.489041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:10:07.489041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:07.491165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 429/844] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:10:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:07.494168 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f69a35-bfc1-4190-8b3e-fdcb602add06 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:07.602034 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:07.686553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6715b-8be7-47bb-aceb-aff3f85f7447 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:07.706495 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f69a35-bfc1-4190-8b3e-fdcb602add06 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:07.717008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:07.759950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1e7c1-1f6e-47f1-8d95-ae364a37c30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:07.761165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1e7c1-1f6e-47f1-8d95-ae364a37c30c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:07.761817 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1e7c1-1f6e-47f1-8d95-ae364a37c30c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:07.762419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1e7c1-1f6e-47f1-8d95-ae364a37c30c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:07.780045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea3a45-d729-4717-b6ba-a8e5cc6d1888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:07.782692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea3a45-d729-4717-b6ba-a8e5cc6d1888 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:07.783440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea3a45-d729-4717-b6ba-a8e5cc6d1888 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:07.784110 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea3a45-d729-4717-b6ba-a8e5cc6d1888 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:07.785392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea3a45-d729-4717-b6ba-a8e5cc6d1888 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:07.786117 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea3a45-d729-4717-b6ba-a8e5cc6d1888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:07.787186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1e7c1-1f6e-47f1-8d95-ae364a37c30c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:07.789269 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea3a45-d729-4717-b6ba-a8e5cc6d1888 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:07.789963 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1e7c1-1f6e-47f1-8d95-ae364a37c30c None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:07.806672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 430/845] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:10:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:07.815819 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea3a45-d729-4717-b6ba-a8e5cc6d1888 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:07.825407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-e3c1e7c1-1f6e-47f1-8d95-ae364a37c30c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:07.825407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:07.825407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:07.825407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:07.825407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:07.825407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:07.825407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:07.825407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:07.825407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:07.825407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:07.825407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:07.825407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:07.825407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:07.825407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:07.825407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:07.825407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:07.827233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:07.827233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:07.827233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:07.827233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:07.827233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:07.827233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:07.827233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:07.827233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:07.827233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:07.827233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:07.827233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:07.827233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:07.827233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:07.827233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:07.827233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:07.827233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:07.827233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:07.831297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:07.831297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:07.831297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:07.835380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 416/846] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:07.846957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1e7c1-1f6e-47f1-8d95-ae364a37c30c None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:08.013653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1e7c1-1f6e-47f1-8d95-ae364a37c30c None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:08.033909 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea3a45-d729-4717-b6ba-a8e5cc6d1888 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:08.059558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:08.072968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f229cd-ee0f-4202-b569-93cb219a71e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:08.074565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f229cd-ee0f-4202-b569-93cb219a71e8 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:08.075368 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f229cd-ee0f-4202-b569-93cb219a71e8 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:08.083881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f229cd-ee0f-4202-b569-93cb219a71e8 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:08.083881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f229cd-ee0f-4202-b569-93cb219a71e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:08.083881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f229cd-ee0f-4202-b569-93cb219a71e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:08.083881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f229cd-ee0f-4202-b569-93cb219a71e8 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:08.102805 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dd2c5-dd5b-4eba-a846-c3c2818117cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:08.104045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dd2c5-dd5b-4eba-a846-c3c2818117cb None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:08.104626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dd2c5-dd5b-4eba-a846-c3c2818117cb None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:08.105130 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dd2c5-dd5b-4eba-a846-c3c2818117cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:08.105297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 417/847] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:08.117928 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f229cd-ee0f-4202-b569-93cb219a71e8 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:08.127119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7dd2c5-dd5b-4eba-a846-c3c2818117cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:08.129836 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7dd2c5-dd5b-4eba-a846-c3c2818117cb None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:08.156325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-7a7dd2c5-dd5b-4eba-a846-c3c2818117cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:08.156325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:08.156325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:08.156325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:08.156325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:08.156325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:08.156325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:08.156325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:08.156325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:08.156325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:08.156325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:08.156325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:08.156325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:08.156325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:08.156325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:08.156325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:08.158188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:08.158188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:08.158188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:08.158188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:08.158188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:08.158188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:08.158188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:08.158188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:08.158188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:08.158188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:08.158188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:08.158188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:08.158188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:08.158188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:08.158188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:08.158188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:08.158188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:08.159982 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:08.159982 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:08.159982 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:08.164068 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 431/848] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:08.168650 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dd2c5-dd5b-4eba-a846-c3c2818117cb None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:08.172104 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:08.320987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f229cd-ee0f-4202-b569-93cb219a71e8 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:08.360158 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dd2c5-dd5b-4eba-a846-c3c2818117cb None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:08.375811 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52d5ae-e0a2-4393-8d96-5ef0a16c572d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:08.376782 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52d5ae-e0a2-4393-8d96-5ef0a16c572d None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:08.377074 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52d5ae-e0a2-4393-8d96-5ef0a16c572d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:08.377387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52d5ae-e0a2-4393-8d96-5ef0a16c572d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:08.388344 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52d5ae-e0a2-4393-8d96-5ef0a16c572d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:08.390015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52d5ae-e0a2-4393-8d96-5ef0a16c572d None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:08.406294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-8b52d5ae-e0a2-4393-8d96-5ef0a16c572d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:10:08.406294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:08.406294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:08.406294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:08.406294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:08.406294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:08.406294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:08.406294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:08.406294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:08.406294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:08.406294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:08.406294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:08.406294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:10:08.406294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:10:08.406294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:10:08.406294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:08.408267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:08.410170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:08.410170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:08.410170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:10:08.410170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:10:08.410170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:10:08.410170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:10:08.410170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:10:08.410170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:10:08.410170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:10:08.410170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:08.410170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 418/849] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 7 13:10:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:08.422434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52d5ae-e0a2-4393-8d96-5ef0a16c572d None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:08.438835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df4fdb-363d-4d55-bdf4-808cb7da005f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:08.439953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df4fdb-363d-4d55-bdf4-808cb7da005f None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:08.440433 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df4fdb-363d-4d55-bdf4-808cb7da005f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:08.440922 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df4fdb-363d-4d55-bdf4-808cb7da005f None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:08.442248 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df4fdb-363d-4d55-bdf4-808cb7da005f None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:08.442844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df4fdb-363d-4d55-bdf4-808cb7da005f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:08.446879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df4fdb-363d-4d55-bdf4-808cb7da005f None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:08.484088 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 432/850] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:08.496120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df4fdb-363d-4d55-bdf4-808cb7da005f None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:08.598567 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:08.709014 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:08.744629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52d5ae-e0a2-4393-8d96-5ef0a16c572d None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:08.828625 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287c9cf-8c4a-4798-a6ee-691622b6fabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:08.829612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287c9cf-8c4a-4798-a6ee-691622b6fabd None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:08.830521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287c9cf-8c4a-4798-a6ee-691622b6fabd None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:08.831276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287c9cf-8c4a-4798-a6ee-691622b6fabd None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:08.833145 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287c9cf-8c4a-4798-a6ee-691622b6fabd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:08.839479 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287c9cf-8c4a-4798-a6ee-691622b6fabd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:08.842640 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287c9cf-8c4a-4798-a6ee-691622b6fabd None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:08.858455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df4fdb-363d-4d55-bdf4-808cb7da005f None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:08.943738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 419/851] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Sep 7 13:10:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 523 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:08.950800 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89bdd8d-93c8-4d08-ba6e-42a367d8f475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:08.952079 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bdd8d-93c8-4d08-ba6e-42a367d8f475 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:08.953835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bdd8d-93c8-4d08-ba6e-42a367d8f475 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:08.954343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bdd8d-93c8-4d08-ba6e-42a367d8f475 None admin] PATH_INFO: `/v3/users/swift` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:08.957424 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287c9cf-8c4a-4798-a6ee-691622b6fabd None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:08.981887 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89bdd8d-93c8-4d08-ba6e-42a367d8f475 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:08.991257 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89bdd8d-93c8-4d08-ba6e-42a367d8f475 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:09.040473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-d89bdd8d-93c8-4d08-ba6e-42a367d8f475 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:10:09.040473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:09.040473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:09.040473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:09.040473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:09.040473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:09.040473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:09.040473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:09.040473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:09.040473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:09.040473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:09.040473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:09.040473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:10:09.040473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:10:09.040473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:10:09.040473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:09.042308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:09.045230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:09.045230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:09.045230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:10:09.045230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:10:09.045230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:10:09.045230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:10:09.045230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:10:09.045230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:10:09.045230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 13:10:09.045230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:09.051474 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 433/852] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:10:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 555 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:09.061387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89bdd8d-93c8-4d08-ba6e-42a367d8f475 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:09.300140 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287c9cf-8c4a-4798-a6ee-691622b6fabd None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:09.342854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:09.372046 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89bdd8d-93c8-4d08-ba6e-42a367d8f475 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:09.423719 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd187dd-d185-48b0-bfa6-70f7f0b76d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:09.424844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd187dd-d185-48b0-bfa6-70f7f0b76d5f None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:09.425458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd187dd-d185-48b0-bfa6-70f7f0b76d5f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:09.425938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd187dd-d185-48b0-bfa6-70f7f0b76d5f None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:09.427041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd187dd-d185-48b0-bfa6-70f7f0b76d5f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:09.427672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd187dd-d185-48b0-bfa6-70f7f0b76d5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:09.429574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd187dd-d185-48b0-bfa6-70f7f0b76d5f None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:09.445058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28075b-95ff-4c1e-98fb-09743ba4bfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:09.446122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28075b-95ff-4c1e-98fb-09743ba4bfb2 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:09.446591 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28075b-95ff-4c1e-98fb-09743ba4bfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:09.453715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28075b-95ff-4c1e-98fb-09743ba4bfb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:09.461574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 434/853] 10.222.0.62 () {56 vars in 1135 bytes} [Wed Sep 7 13:10:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:09.463148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd187dd-d185-48b0-bfa6-70f7f0b76d5f None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:09.553663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:09.603586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd187dd-d185-48b0-bfa6-70f7f0b76d5f None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:09.646640 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28075b-95ff-4c1e-98fb-09743ba4bfb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:09.648680 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28075b-95ff-4c1e-98fb-09743ba4bfb2 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:09.655937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a95ee-c12d-4678-87a6-bd80492498eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:09.656742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a95ee-c12d-4678-87a6-bd80492498eb None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:09.657125 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a95ee-c12d-4678-87a6-bd80492498eb None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:09.657598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a95ee-c12d-4678-87a6-bd80492498eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:09.687043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:09.783991 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a95ee-c12d-4678-87a6-bd80492498eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:09.786811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a95ee-c12d-4678-87a6-bd80492498eb None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:09.848141 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 420/854] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:10:08 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 892 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:10:09.851471 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20662586-aea0-4c43-b1e0-00d5575686a0 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:09.852356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20662586-aea0-4c43-b1e0-00d5575686a0 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:09.853115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20662586-aea0-4c43-b1e0-00d5575686a0 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:09.863367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 421/855] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:09 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:09.863367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20662586-aea0-4c43-b1e0-00d5575686a0 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:09.950865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:09.999037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 435/856] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:10:09 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:10:10.001765 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bbff7-aed2-4fd6-961b-803ec2e76d47 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:10.002245 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bbff7-aed2-4fd6-961b-803ec2e76d47 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:10.002721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bbff7-aed2-4fd6-961b-803ec2e76d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:10.028114 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b86bbff7-aed2-4fd6-961b-803ec2e76d47 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:10.029773 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b86bbff7-aed2-4fd6-961b-803ec2e76d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:10.049446 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20662586-aea0-4c43-b1e0-00d5575686a0 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:10.093033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:10.122764 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399bd1d-8740-415f-b746-fd774aef4cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:10.123814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399bd1d-8740-415f-b746-fd774aef4cdf None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:10.124783 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399bd1d-8740-415f-b746-fd774aef4cdf None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:10.125266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399bd1d-8740-415f-b746-fd774aef4cdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:10.146888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399bd1d-8740-415f-b746-fd774aef4cdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:10.149790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399bd1d-8740-415f-b746-fd774aef4cdf None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:10.159372 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 436/857] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:10.159372 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86bbff7-aed2-4fd6-961b-803ec2e76d47 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:10.190108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-d399bd1d-8740-415f-b746-fd774aef4cdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:10.190108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:10.190108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:10.190108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:10.190108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:10.190108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:10.190108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:10.190108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:10.190108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:10.190108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:10.190108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:10.190108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:10.190108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:10.190108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:10.190108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:10.190108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:10.191790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:10.191790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:10.191790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:10.191790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:10.191790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:10.191790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:10.191790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:10.191790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:10.191790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:10.191790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:10.191790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:10.191790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:10.191790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:10.191790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:10.191790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:10.191790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:10.191790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:10.193279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:10.193279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:10.193279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:10.199583 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 422/858] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:10.201286 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399bd1d-8740-415f-b746-fd774aef4cdf None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:10.404700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86bbff7-aed2-4fd6-961b-803ec2e76d47 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:10.424560 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:10.432075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399bd1d-8740-415f-b746-fd774aef4cdf None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:10.494803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570e284-59af-4372-919c-b7e0ef361c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:10.495436 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570e284-59af-4372-919c-b7e0ef361c88 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:10.495722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570e284-59af-4372-919c-b7e0ef361c88 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:10.496055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570e284-59af-4372-919c-b7e0ef361c88 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:10.496788 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570e284-59af-4372-919c-b7e0ef361c88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:10.497097 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570e284-59af-4372-919c-b7e0ef361c88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:10.498540 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570e284-59af-4372-919c-b7e0ef361c88 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:10.504580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114f5b4-049d-44d8-b309-f307a75531c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:10.506130 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114f5b4-049d-44d8-b309-f307a75531c5 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:10.506940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114f5b4-049d-44d8-b309-f307a75531c5 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:10.507761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114f5b4-049d-44d8-b309-f307a75531c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:10.516945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 423/859] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:10.518966 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2975eed-faa4-4dc9-879b-a6f0b5f0db4f None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:10.519650 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2975eed-faa4-4dc9-879b-a6f0b5f0db4f None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:10.520171 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2975eed-faa4-4dc9-879b-a6f0b5f0db4f None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:10.522117 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 424/860] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:10.529665 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2975eed-faa4-4dc9-879b-a6f0b5f0db4f None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:10.532463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6114f5b4-049d-44d8-b309-f307a75531c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:10.535282 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6114f5b4-049d-44d8-b309-f307a75531c5 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:10.569734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-6114f5b4-049d-44d8-b309-f307a75531c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:10.569734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:10.569734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:10.569734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:10.569734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:10.569734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:10.569734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:10.569734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:10.569734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:10.569734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:10.569734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:10.569734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:10.569734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:10.569734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:10.569734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:10.569734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:10.571402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:10.571402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:10.571402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:10.571402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:10.571402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:10.571402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:10.571402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:10.571402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:10.571402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:10.571402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:10.571402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:10.571402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:10.571402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:10.571402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:10.571402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:10.571402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:10.571402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:10.573770 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:10.573770 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:10.573770 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:10.573770 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 437/861] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:10.573770 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd5eac-bd4d-4a0c-9e79-a9aad32d268d None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:10.573770 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd5eac-bd4d-4a0c-9e79-a9aad32d268d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:10.573770 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd5eac-bd4d-4a0c-9e79-a9aad32d268d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:10.602849 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-16cd5eac-bd4d-4a0c-9e79-a9aad32d268d None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:10.604434 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-16cd5eac-bd4d-4a0c-9e79-a9aad32d268d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:10.636086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:10.721209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 438/862] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:10.722011 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd5eac-bd4d-4a0c-9e79-a9aad32d268d None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:10.734866 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2975eed-faa4-4dc9-879b-a6f0b5f0db4f None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:10.798312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18a131-88bc-427c-9466-8d16ff0ffd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:10.799428 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18a131-88bc-427c-9466-8d16ff0ffd3e None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:10.799913 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18a131-88bc-427c-9466-8d16ff0ffd3e None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:10.800386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18a131-88bc-427c-9466-8d16ff0ffd3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:10.815949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18a131-88bc-427c-9466-8d16ff0ffd3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:10.818966 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18a131-88bc-427c-9466-8d16ff0ffd3e None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:10.843893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-9b18a131-88bc-427c-9466-8d16ff0ffd3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:10.843893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:10.843893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:10.843893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:10.843893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:10.843893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:10.843893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:10.843893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:10.843893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:10.843893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:10.843893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:10.843893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:10.843893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:10:10.843893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:10:10.843893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:10:10.844957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:10:10.844957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:10.844957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:10.844957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:10.844957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:10.844957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:10.844957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:10.844957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:10.844957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:10.844957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:10.844957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:10.844957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:10.844957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:10.844957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:10.844957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:10.844957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:10.844957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:10.846131 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:10:10.846131 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:10:10.846131 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:10:10.846131 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:10:10.846131 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:10:10.846131 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:10:10.846131 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:10.846131 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:10.846579 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 425/863] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:10:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:10.847864 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18a131-88bc-427c-9466-8d16ff0ffd3e None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:10.889479 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:10.954136 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd5eac-bd4d-4a0c-9e79-a9aad32d268d None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:11.020286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3b195-c55d-42bd-b2d0-272b381fd71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:11.021189 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3b195-c55d-42bd-b2d0-272b381fd71e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:11.021667 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3b195-c55d-42bd-b2d0-272b381fd71e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:11.022159 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3b195-c55d-42bd-b2d0-272b381fd71e None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:11.023438 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3b195-c55d-42bd-b2d0-272b381fd71e None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:11.024304 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3b195-c55d-42bd-b2d0-272b381fd71e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:11.026551 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3b195-c55d-42bd-b2d0-272b381fd71e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:11.058228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 439/864] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:11.058228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3b195-c55d-42bd-b2d0-272b381fd71e None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:11.068997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18a131-88bc-427c-9466-8d16ff0ffd3e None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:11.098746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:11.132146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b064754-151f-4288-9d10-ca3aedc9bf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:11.133378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b064754-151f-4288-9d10-ca3aedc9bf0f None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:11.133812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b064754-151f-4288-9d10-ca3aedc9bf0f None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:11.134152 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b064754-151f-4288-9d10-ca3aedc9bf0f None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:11.135281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b064754-151f-4288-9d10-ca3aedc9bf0f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:11.136934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b064754-151f-4288-9d10-ca3aedc9bf0f None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:11.232231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 426/865] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 7 13:10:10 2022] POST /identity/v3/services => generated 251 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:10:11.233789 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b064754-151f-4288-9d10-ca3aedc9bf0f None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:11.264330 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3b195-c55d-42bd-b2d0-272b381fd71e None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:11.355716 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc8f28-7658-45fb-9b16-986dbfe9058c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:11.356781 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc8f28-7658-45fb-9b16-986dbfe9058c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:11.357270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc8f28-7658-45fb-9b16-986dbfe9058c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:11.357782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc8f28-7658-45fb-9b16-986dbfe9058c None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:11.359700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc8f28-7658-45fb-9b16-986dbfe9058c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:11.360446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc8f28-7658-45fb-9b16-986dbfe9058c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:11.363255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc8f28-7658-45fb-9b16-986dbfe9058c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:11.396864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 440/866] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:10:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:11.396864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc8f28-7658-45fb-9b16-986dbfe9058c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:11.404645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:11.465681 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b064754-151f-4288-9d10-ca3aedc9bf0f None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:11.547319 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65f4ba-b898-4c3a-a751-8d153aa67d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:11.548227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65f4ba-b898-4c3a-a751-8d153aa67d4c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:11.548979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65f4ba-b898-4c3a-a751-8d153aa67d4c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:11.549184 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65f4ba-b898-4c3a-a751-8d153aa67d4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:11.557175 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:11.569032 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65f4ba-b898-4c3a-a751-8d153aa67d4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:11.574058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65f4ba-b898-4c3a-a751-8d153aa67d4c None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:11.604000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc8f28-7658-45fb-9b16-986dbfe9058c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:11.604460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-bc65f4ba-b898-4c3a-a751-8d153aa67d4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:11.604460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:11.604460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:11.604460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:11.604460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:11.604460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:11.604460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:11.604460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:11.604460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:11.604460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:11.604460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:11.604460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:11.604460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:10:11.604460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:10:11.604460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:10:11.605891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:10:11.605891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:11.605891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:11.605891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:11.605891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:11.605891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:11.605891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:11.605891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:11.605891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:11.605891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:11.605891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:11.605891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:11.605891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:11.605891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:11.605891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:11.605891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:11.605891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:11.606903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:10:11.606903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:10:11.606903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:10:11.606903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:10:11.606903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:10:11.606903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:10:11.606903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:11.606903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:11.615477 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 427/867] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:10:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:11.615477 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65f4ba-b898-4c3a-a751-8d153aa67d4c None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:11.654022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b87989-f521-454b-969c-09e7a6d23751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:11.655040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b87989-f521-454b-969c-09e7a6d23751 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:11.656145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b87989-f521-454b-969c-09e7a6d23751 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:11.656654 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b87989-f521-454b-969c-09e7a6d23751 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:11.657772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b87989-f521-454b-969c-09e7a6d23751 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:11.660242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b87989-f521-454b-969c-09e7a6d23751 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:11.678209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-c4b87989-f521-454b-969c-09e7a6d23751 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:10:11.678209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:11.678209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:11.678209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:11.678209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:11.678209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:11.678209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:11.678209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:11.678209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:10:11.678209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:11.678209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:11.678209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:11.678209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:11.678209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:11.678209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:11.678209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:11.679440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:11.679440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:11.679440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:11.679440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:11.679440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:11.679440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:11.679440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:11.679440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:11.679440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:11.679440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:11.679440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:11.679440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:11.679440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:11.679440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:11.679440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:11.679440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:11.679440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:11.680343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:11.680343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:11.680343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:11.680343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:11.680343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:11.680343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:11.680343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:11.680343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:11.680343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:11.680343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:11.680343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:11.680343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:11.680343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:10:11.680343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:11.683382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 441/868] 10.222.0.62 () {56 vars in 1132 bytes} [Wed Sep 7 13:10:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:11.684748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b87989-f521-454b-969c-09e7a6d23751 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:11.793845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65f4ba-b898-4c3a-a751-8d153aa67d4c None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:11.871259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065543a-9ad2-4795-ad4f-4bdec33d5b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:11.872197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065543a-9ad2-4795-ad4f-4bdec33d5b22 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:11.872698 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065543a-9ad2-4795-ad4f-4bdec33d5b22 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:11.873194 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065543a-9ad2-4795-ad4f-4bdec33d5b22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:11.875536 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065543a-9ad2-4795-ad4f-4bdec33d5b22 None admin] RBAC: Adding query filter params (role.id=743ed2d8e7114975bd9245742a41fe0f, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=8121ba4cf2cf44d9a5a55975ea85928c) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:11.876120 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065543a-9ad2-4795-ad4f-4bdec33d5b22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:11.878283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b87989-f521-454b-969c-09e7a6d23751 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:11.878603 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065543a-9ad2-4795-ad4f-4bdec33d5b22 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:11.884469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:11.889233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 428/869] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:10:11 2022] GET /identity/v3/role_assignments?user.id=8121ba4cf2cf44d9a5a55975ea85928c&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=743ed2d8e7114975bd9245742a41fe0f => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:11.891460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065543a-9ad2-4795-ad4f-4bdec33d5b22 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:11.947953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebffd5-6b4e-4394-9b9c-216000f55928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:11.949431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebffd5-6b4e-4394-9b9c-216000f55928 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:11.949859 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebffd5-6b4e-4394-9b9c-216000f55928 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:11.950342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebffd5-6b4e-4394-9b9c-216000f55928 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:11.953051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebffd5-6b4e-4394-9b9c-216000f55928 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:11.953667 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebffd5-6b4e-4394-9b9c-216000f55928 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:11.957710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebffd5-6b4e-4394-9b9c-216000f55928 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:11.983119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 442/870] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:10:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:11.983639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:11.996823 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebffd5-6b4e-4394-9b9c-216000f55928 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:12.111496 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065543a-9ad2-4795-ad4f-4bdec33d5b22 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:12.191380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f6397-5f29-4a27-b36b-267e9edd0c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:12.192984 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f6397-5f29-4a27-b36b-267e9edd0c76 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:12.193487 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f6397-5f29-4a27-b36b-267e9edd0c76 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:12.194051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f6397-5f29-4a27-b36b-267e9edd0c76 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:12.195269 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f6397-5f29-4a27-b36b-267e9edd0c76 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:12.195932 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f6397-5f29-4a27-b36b-267e9edd0c76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:12.198248 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f6397-5f29-4a27-b36b-267e9edd0c76 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:12.203407 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebffd5-6b4e-4394-9b9c-216000f55928 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:12.215210 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 429/871] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 7 13:10:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:12.218569 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f6397-5f29-4a27-b36b-267e9edd0c76 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:12.267853 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a29d7-ca6c-4b0f-881e-27473c0fec17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:12.268870 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a29d7-ca6c-4b0f-881e-27473c0fec17 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:12.269362 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a29d7-ca6c-4b0f-881e-27473c0fec17 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:12.269870 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a29d7-ca6c-4b0f-881e-27473c0fec17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:12.271501 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a29d7-ca6c-4b0f-881e-27473c0fec17 None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=5cedc7091d2f46b5be68818d26399e9b) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:12.272217 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a29d7-ca6c-4b0f-881e-27473c0fec17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:12.274557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a29d7-ca6c-4b0f-881e-27473c0fec17 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:12.289532 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 443/872] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:10:11 2022] GET /identity/v3/role_assignments?user.id=5cedc7091d2f46b5be68818d26399e9b&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:12.352876 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:12.432541 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f6397-5f29-4a27-b36b-267e9edd0c76 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:12.495672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a8427-b78c-4e95-b6f7-970f53493c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:12.496922 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a8427-b78c-4e95-b6f7-970f53493c94 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:12.497671 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a8427-b78c-4e95-b6f7-970f53493c94 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:12.498396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a8427-b78c-4e95-b6f7-970f53493c94 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:12.499524 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a8427-b78c-4e95-b6f7-970f53493c94 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:12.500204 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a8427-b78c-4e95-b6f7-970f53493c94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:12.502124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a8427-b78c-4e95-b6f7-970f53493c94 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:12.512029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 430/873] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 7 13:10:12 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:15.105721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4a02a-d5d8-48d3-9d67-7d5b6be7ff8f None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:15.107433 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4a02a-d5d8-48d3-9d67-7d5b6be7ff8f None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:15.108237 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4a02a-d5d8-48d3-9d67-7d5b6be7ff8f None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:15.111837 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 444/874] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:15.121802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b325754-f99a-4093-a51c-f520cddd7908 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:15.123556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b325754-f99a-4093-a51c-f520cddd7908 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:15.124459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b325754-f99a-4093-a51c-f520cddd7908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:15.160086 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-7b325754-f99a-4093-a51c-f520cddd7908 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:15.162887 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-7b325754-f99a-4093-a51c-f520cddd7908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:15.310809 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 431/875] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:15 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:10:15.318023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a612bf3-c120-44f3-857a-d946087fef56 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:15.320820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a612bf3-c120-44f3-857a-d946087fef56 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:15.320820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a612bf3-c120-44f3-857a-d946087fef56 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:15.320820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 445/876] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:10:15.328801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bccc8d-660c-450c-a5f4-6227600a0c11 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:15.329725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bccc8d-660c-450c-a5f4-6227600a0c11 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:15.330730 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bccc8d-660c-450c-a5f4-6227600a0c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:15.364392 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-a5bccc8d-660c-450c-a5f4-6227600a0c11 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:15.367055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a5bccc8d-660c-450c-a5f4-6227600a0c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:15.424906 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fccdb8-58b0-430f-9743-4a012f91d993 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:15.425204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fccdb8-58b0-430f-9743-4a012f91d993 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:15.425534 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fccdb8-58b0-430f-9743-4a012f91d993 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:15.428139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 446/877] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:15.437882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6f77a-c5b3-494c-bb85-1c4e47a07dd5 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:15.438372 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6f77a-c5b3-494c-bb85-1c4e47a07dd5 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:15.438854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6f77a-c5b3-494c-bb85-1c4e47a07dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:15.485039 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-48a6f77a-c5b3-494c-bb85-1c4e47a07dd5 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:15.487384 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-48a6f77a-c5b3-494c-bb85-1c4e47a07dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:15.489816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:15.527629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 432/878] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:15 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:15.609805 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:15.625390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 447/879] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:15 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:10:15.763644 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bccc8d-660c-450c-a5f4-6227600a0c11 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:15.929893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bccc8d-660c-450c-a5f4-6227600a0c11 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:15.978254 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6f77a-c5b3-494c-bb85-1c4e47a07dd5 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:16.011219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e61605-294c-47b0-8f10-a73f9a3c3e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:16.012316 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e61605-294c-47b0-8f10-a73f9a3c3e48 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:16.012802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e61605-294c-47b0-8f10-a73f9a3c3e48 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:16.013264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e61605-294c-47b0-8f10-a73f9a3c3e48 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:16.014238 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e61605-294c-47b0-8f10-a73f9a3c3e48 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:16.016311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e61605-294c-47b0-8f10-a73f9a3c3e48 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:16.035358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-63e61605-294c-47b0-8f10-a73f9a3c3e48 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:10:16.035358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:16.035358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:16.035358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:16.035358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:16.035358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:16.035358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:16.035358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:16.035358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:10:16.035358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:16.035358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:16.035358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:16.035358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:16.035358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:16.035358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:16.035358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:16.035358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:16.041676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:16.041676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:16.041676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:16.041676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:16.041676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:16.041676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:16.041676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:16.041676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:16.041676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:16.041676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:16.041676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:16.041676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:16.041676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:16.041676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:16.041676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:16.041676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:16.044151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:16.044151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:16.044151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:16.044151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:16.044151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:16.044151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:16.044151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:16.044151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:16.044151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:16.044151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:16.044151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:16.044151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:16.044151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:10:16.044151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:16.044151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 433/880] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:10:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:16.049464 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e61605-294c-47b0-8f10-a73f9a3c3e48 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:16.148631 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:16.209944 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6f77a-c5b3-494c-bb85-1c4e47a07dd5 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:16.244174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e61605-294c-47b0-8f10-a73f9a3c3e48 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:16.262123 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c65952-4d78-46b9-ab0c-245dbe91e2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:16.262798 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c65952-4d78-46b9-ab0c-245dbe91e2bc None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:16.263291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c65952-4d78-46b9-ab0c-245dbe91e2bc None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:16.263668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c65952-4d78-46b9-ab0c-245dbe91e2bc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:16.282778 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c65952-4d78-46b9-ab0c-245dbe91e2bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:10:16.292881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-12c65952-4d78-46b9-ab0c-245dbe91e2bc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:10:16.292881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:16.292881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:16.292881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:16.292881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:16.292881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:16.292881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:16.292881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:16.292881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:16.292881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:16.292881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:16.292881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:16.292881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:10:16.292881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:10:16.292881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:10:16.292881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:10:16.294763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:10:16.294763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:10:16.294763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:10:16.294763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:10:16.294763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:16.294763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:16.294763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:16.294763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:16.294763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:16.294763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:16.294763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:16.294763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:16.294763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:16.294763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:16.294763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:16.294763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:16.296629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:16.296629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:16.296629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:16.296629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:16.296629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:10:16.296629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:10:16.296629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:10:16.296629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:10:16.296629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:10:16.296629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:10:16.296629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:10:16.296629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:16.296629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 448/881] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Sep 7 13:10:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:16.296629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c65952-4d78-46b9-ab0c-245dbe91e2bc None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:16.308782 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dfa14c-9afe-4318-ae6a-011156c6cfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:16.309446 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfa14c-9afe-4318-ae6a-011156c6cfff None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:16.309727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfa14c-9afe-4318-ae6a-011156c6cfff None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:16.310000 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfa14c-9afe-4318-ae6a-011156c6cfff None admin] PATH_INFO: `/v3/services/object-store` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:16.310609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dfa14c-9afe-4318-ae6a-011156c6cfff None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:16.312832 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dfa14c-9afe-4318-ae6a-011156c6cfff None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:16.323339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-10dfa14c-9afe-4318-ae6a-011156c6cfff None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:10:16.323339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:16.323339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:16.323339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:16.323339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:16.323339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:16.323339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:16.323339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:16.323339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:10:16.323339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:16.323339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:16.323339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:16.323339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:16.323339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:16.323339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:16.323339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:16.323339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:16.324486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:16.324486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:16.324486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:16.324486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:16.324486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:16.324486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:16.324486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:16.324486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:16.324486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:16.324486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:16.324486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:16.324486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:16.324486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:16.324486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:16.324486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:16.324486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:16.325590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:16.325590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:16.325590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:16.325590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:16.325590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:16.325590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:16.325590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:16.325590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:16.325590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:16.325590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:16.325590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:16.325590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:16.325590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:10:16.325590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:16.325590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 434/882] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 7 13:10:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:16.326471 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530257c7-0295-4f9e-896e-db480da8c5b3 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:16.326471 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530257c7-0295-4f9e-896e-db480da8c5b3 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:16.326607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530257c7-0295-4f9e-896e-db480da8c5b3 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:16.328783 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 435/883] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:16.329415 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530257c7-0295-4f9e-896e-db480da8c5b3 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:16.379353 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:16.470239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c65952-4d78-46b9-ab0c-245dbe91e2bc None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:16.498693 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530257c7-0295-4f9e-896e-db480da8c5b3 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:16.527177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fc5fe-54a8-4b0a-9b41-5ea801cc566d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:16.528397 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fc5fe-54a8-4b0a-9b41-5ea801cc566d None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:16.529021 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fc5fe-54a8-4b0a-9b41-5ea801cc566d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:16.529669 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fc5fe-54a8-4b0a-9b41-5ea801cc566d None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:16.531744 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5fc5fe-54a8-4b0a-9b41-5ea801cc566d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:16.532123 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5fc5fe-54a8-4b0a-9b41-5ea801cc566d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:16.533281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:16.534222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5fc5fe-54a8-4b0a-9b41-5ea801cc566d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:16.541256 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 449/884] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:10:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:16.542303 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fc5fe-54a8-4b0a-9b41-5ea801cc566d None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:16.565325 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6467852-c55d-4ccc-8554-d907afd472b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:16.566621 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6467852-c55d-4ccc-8554-d907afd472b0 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:16.567166 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6467852-c55d-4ccc-8554-d907afd472b0 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:16.567641 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6467852-c55d-4ccc-8554-d907afd472b0 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:16.568876 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6467852-c55d-4ccc-8554-d907afd472b0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:16.569411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6467852-c55d-4ccc-8554-d907afd472b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:16.572481 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6467852-c55d-4ccc-8554-d907afd472b0 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:16.594443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 436/885] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Sep 7 13:10:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:16.594443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d5f64-e655-421b-b8e8-0d0aa7bc4f5f None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:16.594443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d5f64-e655-421b-b8e8-0d0aa7bc4f5f None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:16.594909 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d5f64-e655-421b-b8e8-0d0aa7bc4f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:16.627399 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-a58d5f64-e655-421b-b8e8-0d0aa7bc4f5f None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:16.629188 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a58d5f64-e655-421b-b8e8-0d0aa7bc4f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:16.725872 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fc5fe-54a8-4b0a-9b41-5ea801cc566d None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:16.744372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 437/886] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:16.745885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d5f64-e655-421b-b8e8-0d0aa7bc4f5f None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:16.751342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:16.797831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce88d96-3612-42ee-97c3-06cdd02d8c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:16.798932 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce88d96-3612-42ee-97c3-06cdd02d8c95 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:16.800261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce88d96-3612-42ee-97c3-06cdd02d8c95 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:16.801157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce88d96-3612-42ee-97c3-06cdd02d8c95 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:16.802408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce88d96-3612-42ee-97c3-06cdd02d8c95 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:16.802954 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce88d96-3612-42ee-97c3-06cdd02d8c95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:16.806440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce88d96-3612-42ee-97c3-06cdd02d8c95 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:16.826152 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 450/887] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 7 13:10:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:16.826152 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce88d96-3612-42ee-97c3-06cdd02d8c95 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:16.978285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d5f64-e655-421b-b8e8-0d0aa7bc4f5f None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:16.997890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:17.000427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce88d96-3612-42ee-97c3-06cdd02d8c95 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:17.037500 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ffa65-b43b-4bd2-8542-582daab88f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:17.038157 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ffa65-b43b-4bd2-8542-582daab88f11 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:17.038458 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ffa65-b43b-4bd2-8542-582daab88f11 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:17.038787 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ffa65-b43b-4bd2-8542-582daab88f11 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:17.039948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ffa65-b43b-4bd2-8542-582daab88f11 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:17.040458 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ffa65-b43b-4bd2-8542-582daab88f11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:17.042480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ffa65-b43b-4bd2-8542-582daab88f11 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:17.057695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 438/888] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:10:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:17.057695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ffa65-b43b-4bd2-8542-582daab88f11 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:17.069037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96137cb2-8756-4b4c-868c-cb532c7bb5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:17.069717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96137cb2-8756-4b4c-868c-cb532c7bb5ac None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:17.070115 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96137cb2-8756-4b4c-868c-cb532c7bb5ac None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:17.070527 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96137cb2-8756-4b4c-868c-cb532c7bb5ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:17.092594 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96137cb2-8756-4b4c-868c-cb532c7bb5ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:17.095982 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96137cb2-8756-4b4c-868c-cb532c7bb5ac None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:17.117963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-96137cb2-8756-4b4c-868c-cb532c7bb5ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:17.117963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:17.117963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:17.117963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:17.117963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:17.117963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:17.117963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:17.117963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:17.117963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:17.117963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:17.117963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:17.117963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:17.117963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:17.117963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:17.117963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:17.117963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:17.119668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:17.119668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:17.119668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:17.119668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:17.119668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:17.119668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:17.119668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:17.119668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:17.119668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:17.119668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:17.119668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:17.119668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:17.119668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:17.119668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:17.119668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:17.119668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:17.119668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:17.121430 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:17.121430 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:17.121430 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:17.121430 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 451/889] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:17.121430 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96137cb2-8756-4b4c-868c-cb532c7bb5ac None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:17.179718 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:17.223912 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ffa65-b43b-4bd2-8542-582daab88f11 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:17.270387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e51955-196b-4fed-b9b6-687d1319e241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:17.270943 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e51955-196b-4fed-b9b6-687d1319e241 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:17.271244 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e51955-196b-4fed-b9b6-687d1319e241 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:17.271532 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e51955-196b-4fed-b9b6-687d1319e241 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:17.272253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e51955-196b-4fed-b9b6-687d1319e241 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:17.272602 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e51955-196b-4fed-b9b6-687d1319e241 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:17.274145 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e51955-196b-4fed-b9b6-687d1319e241 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:17.281700 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 439/890] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 7 13:10:17 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:17.282807 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e51955-196b-4fed-b9b6-687d1319e241 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:17.332381 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:17.345237 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96137cb2-8756-4b4c-868c-cb532c7bb5ac None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:17.418037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791c6f6-8789-4fb6-a11f-9935820a5297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:17.419291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791c6f6-8789-4fb6-a11f-9935820a5297 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:17.419729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791c6f6-8789-4fb6-a11f-9935820a5297 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:17.420169 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791c6f6-8789-4fb6-a11f-9935820a5297 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:17.422511 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791c6f6-8789-4fb6-a11f-9935820a5297 None admin] RBAC: Adding query filter params (interface=public, service_id=50509ff865ba4479b310853def10af00, region_id=RegionOne) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:17.423039 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791c6f6-8789-4fb6-a11f-9935820a5297 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:17.425527 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791c6f6-8789-4fb6-a11f-9935820a5297 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:17.435066 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 452/891] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Sep 7 13:10:17 2022] GET /identity/v3/endpoints?service_id=50509ff865ba4479b310853def10af00&interface=public®ion_id=RegionOne => generated 194 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:17.436705 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791c6f6-8789-4fb6-a11f-9935820a5297 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:17.452784 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e51955-196b-4fed-b9b6-687d1319e241 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:17.509326 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8b306-00c6-496e-97c6-c2b6d122d040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:17.509999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8b306-00c6-496e-97c6-c2b6d122d040 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:17.510391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8b306-00c6-496e-97c6-c2b6d122d040 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:17.510863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8b306-00c6-496e-97c6-c2b6d122d040 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:17.511974 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8b306-00c6-496e-97c6-c2b6d122d040 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:17.512320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8b306-00c6-496e-97c6-c2b6d122d040 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:17.514211 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8b306-00c6-496e-97c6-c2b6d122d040 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:17.539547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 440/892] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:17.542277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8b306-00c6-496e-97c6-c2b6d122d040 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:17.630349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:17.645380 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791c6f6-8789-4fb6-a11f-9935820a5297 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:17.713136 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebff64-622c-4200-af98-3d32d64da4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:17.714218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebff64-622c-4200-af98-3d32d64da4c0 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:17.714750 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebff64-622c-4200-af98-3d32d64da4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:17.715528 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebff64-622c-4200-af98-3d32d64da4c0 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:17.716700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebff64-622c-4200-af98-3d32d64da4c0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:17.720092 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebff64-622c-4200-af98-3d32d64da4c0 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:17.740340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:17.751421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8b306-00c6-496e-97c6-c2b6d122d040 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:17.819138 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 453/893] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 7 13:10:17 2022] POST /identity/v3/services => generated 269 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:10:17.819138 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebff64-622c-4200-af98-3d32d64da4c0 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:17.826093 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5f6e5-f383-407d-b53a-23bdbdf84531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:17.827833 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5f6e5-f383-407d-b53a-23bdbdf84531 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:17.828703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5f6e5-f383-407d-b53a-23bdbdf84531 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:17.829219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5f6e5-f383-407d-b53a-23bdbdf84531 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:17.830677 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5f6e5-f383-407d-b53a-23bdbdf84531 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:17.833729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5f6e5-f383-407d-b53a-23bdbdf84531 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:17.850112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 441/894] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 7 13:10:17 2022] GET /identity/v3/services => generated 1268 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:10:18.060689 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebff64-622c-4200-af98-3d32d64da4c0 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:18.138536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ed1ee-12e7-4742-88e9-f0f2eddaba5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:18.140040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ed1ee-12e7-4742-88e9-f0f2eddaba5b None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:18.140516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ed1ee-12e7-4742-88e9-f0f2eddaba5b None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:18.140895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ed1ee-12e7-4742-88e9-f0f2eddaba5b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:18.158084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ed1ee-12e7-4742-88e9-f0f2eddaba5b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:18.170541 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ed1ee-12e7-4742-88e9-f0f2eddaba5b None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:18.194513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-9c7ed1ee-12e7-4742-88e9-f0f2eddaba5b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:10:18.194513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:18.194513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:18.194513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:18.194513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:18.194513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:18.194513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:18.194513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:18.194513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:18.194513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:18.194513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:18.194513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:18.194513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:10:18.194513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:10:18.194513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:10:18.194513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:18.196440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:18.198210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:18.198210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:18.198210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:10:18.198210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:10:18.198210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:10:18.198210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:10:18.198210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:10:18.198210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:10:18.198210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:10:18.198210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:18.198210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 454/895] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 7 13:10:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:18.209392 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5f6e5-f383-407d-b53a-23bdbdf84531 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:18.426505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5f6e5-f383-407d-b53a-23bdbdf84531 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:18.497774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e2179-0f72-4951-9f4e-d43ef3666604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:18.498827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e2179-0f72-4951-9f4e-d43ef3666604 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:18.498827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e2179-0f72-4951-9f4e-d43ef3666604 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:18.498827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e2179-0f72-4951-9f4e-d43ef3666604 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:18.501722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e2179-0f72-4951-9f4e-d43ef3666604 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:18.502251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e2179-0f72-4951-9f4e-d43ef3666604 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:18.505196 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e2179-0f72-4951-9f4e-d43ef3666604 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:18.542138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 442/896] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Sep 7 13:10:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:18.556733 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ed1ee-12e7-4742-88e9-f0f2eddaba5b None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:18.591590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:18.766023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ed1ee-12e7-4742-88e9-f0f2eddaba5b None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:18.835657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859f7b8-7517-4df8-a683-932e294b79af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:18.839332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859f7b8-7517-4df8-a683-932e294b79af None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:18.839801 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859f7b8-7517-4df8-a683-932e294b79af None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:18.841883 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859f7b8-7517-4df8-a683-932e294b79af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:18.947938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859f7b8-7517-4df8-a683-932e294b79af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:18.949576 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859f7b8-7517-4df8-a683-932e294b79af None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:18.967418 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:19.091543 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 455/897] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:10:18 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:10:19.102033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e2179-0f72-4951-9f4e-d43ef3666604 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:19.139417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:19.326409 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e2179-0f72-4951-9f4e-d43ef3666604 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:19.397242 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f6c36-fd68-4797-af17-0af5e1d9d146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:19.398194 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f6c36-fd68-4797-af17-0af5e1d9d146 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:19.398749 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f6c36-fd68-4797-af17-0af5e1d9d146 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:19.399274 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f6c36-fd68-4797-af17-0af5e1d9d146 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:19.418530 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f6c36-fd68-4797-af17-0af5e1d9d146 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:19.422012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f6c36-fd68-4797-af17-0af5e1d9d146 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:19.453547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-095f6c36-fd68-4797-af17-0af5e1d9d146 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:19.453547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:19.453547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:19.453547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:19.453547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:19.453547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:19.453547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:19.453547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:19.453547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:19.453547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:19.453547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:19.453547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:19.453547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:19.453547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:19.453547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:19.453547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:19.456552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:19.456552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:19.456552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:19.456552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:19.456552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:19.456552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:19.456552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:19.456552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:19.456552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:19.456552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:19.456552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:19.456552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:19.456552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:19.456552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:19.456552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:19.456552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:19.456552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:19.458447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:19.458447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:19.458447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:19.458447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 443/898] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:19.470738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859f7b8-7517-4df8-a683-932e294b79af None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:19.637543 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859f7b8-7517-4df8-a683-932e294b79af None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:19.657745 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:19.685507 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aded6df-d4f2-490e-883b-bc3720821714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:19.686712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aded6df-d4f2-490e-883b-bc3720821714 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:19.687337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aded6df-d4f2-490e-883b-bc3720821714 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:19.687974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aded6df-d4f2-490e-883b-bc3720821714 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:19.688929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aded6df-d4f2-490e-883b-bc3720821714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:19.689558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aded6df-d4f2-490e-883b-bc3720821714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:19.691266 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aded6df-d4f2-490e-883b-bc3720821714 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:19.709749 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 456/899] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:19.719821 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f6c36-fd68-4797-af17-0af5e1d9d146 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:19.844193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:19.895251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f6c36-fd68-4797-af17-0af5e1d9d146 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:19.941152 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48fc0c-326f-4ecd-918b-5717ac6e5adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:19.942186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48fc0c-326f-4ecd-918b-5717ac6e5adf None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:19.942722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48fc0c-326f-4ecd-918b-5717ac6e5adf None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:19.943288 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48fc0c-326f-4ecd-918b-5717ac6e5adf None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:19.952021 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48fc0c-326f-4ecd-918b-5717ac6e5adf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:19.954011 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48fc0c-326f-4ecd-918b-5717ac6e5adf None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:19.970054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-eb48fc0c-326f-4ecd-918b-5717ac6e5adf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:19.970054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:19.970054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:19.970054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:19.970054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:19.970054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:19.970054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:19.970054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:19.970054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:19.970054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:19.970054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:19.970054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:19.970054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:10:19.970054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:10:19.970054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:10:19.971107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:10:19.971107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:19.971107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:19.971107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:19.971107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:19.971107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:19.971107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:19.971107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:19.971107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:19.971107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:19.971107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:19.971107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:19.971107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:19.971107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:19.971107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:19.971107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:19.971107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:19.972180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:10:19.972180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:10:19.972180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:10:19.972180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:10:19.972180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:10:19.972180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:10:19.972180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:19.972180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:19.974715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 444/900] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:10:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:19.985079 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aded6df-d4f2-490e-883b-bc3720821714 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:20.200185 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aded6df-d4f2-490e-883b-bc3720821714 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:20.274716 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718a890-863f-4b57-9728-e59830a33090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:20.276315 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718a890-863f-4b57-9728-e59830a33090 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:20.277065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718a890-863f-4b57-9728-e59830a33090 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:20.277748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718a890-863f-4b57-9728-e59830a33090 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:20.279402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718a890-863f-4b57-9728-e59830a33090 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:20.280929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718a890-863f-4b57-9728-e59830a33090 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:20.284260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718a890-863f-4b57-9728-e59830a33090 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:20.312247 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 457/901] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:10:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:20.321956 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48fc0c-326f-4ecd-918b-5717ac6e5adf None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:20.560693 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48fc0c-326f-4ecd-918b-5717ac6e5adf None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:20.631437 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:20.653840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816227e-59f8-4e16-a0a5-e05d303e4f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:20.654957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816227e-59f8-4e16-a0a5-e05d303e4f73 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:20.656138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816227e-59f8-4e16-a0a5-e05d303e4f73 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:20.656632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816227e-59f8-4e16-a0a5-e05d303e4f73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:20.657863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816227e-59f8-4e16-a0a5-e05d303e4f73 None admin] RBAC: Adding query filter params (role.id=e88e2f39fcb94166851f6db7b9f9dfcb, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=8121ba4cf2cf44d9a5a55975ea85928c) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:20.658419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816227e-59f8-4e16-a0a5-e05d303e4f73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:20.661312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816227e-59f8-4e16-a0a5-e05d303e4f73 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:20.673095 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 445/902] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:10:20 2022] GET /identity/v3/role_assignments?user.id=8121ba4cf2cf44d9a5a55975ea85928c&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=e88e2f39fcb94166851f6db7b9f9dfcb => generated 259 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:20.908636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945344d-0de6-4039-bb7e-c3508c5e12a8 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:20.909677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945344d-0de6-4039-bb7e-c3508c5e12a8 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:20.909677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945344d-0de6-4039-bb7e-c3508c5e12a8 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:20.912684 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 458/903] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:20.922786 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40105b-6a6d-4544-973f-714dc3a22a96 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:20.923313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40105b-6a6d-4544-973f-714dc3a22a96 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:20.923796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40105b-6a6d-4544-973f-714dc3a22a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:20.958575 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-9c40105b-6a6d-4544-973f-714dc3a22a96 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:20.960766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9c40105b-6a6d-4544-973f-714dc3a22a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:21.109462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 446/904] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:21.130590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b5857-6fe2-4982-b08a-df8ee9360d61 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:21.130896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b5857-6fe2-4982-b08a-df8ee9360d61 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:21.131284 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b5857-6fe2-4982-b08a-df8ee9360d61 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:21.133560 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 459/905] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:21.141846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7d728-5521-4db3-9c37-e407c96a187c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:21.142350 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7d728-5521-4db3-9c37-e407c96a187c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:21.142841 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7d728-5521-4db3-9c37-e407c96a187c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:21.178350 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-52e7d728-5521-4db3-9c37-e407c96a187c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:21.180178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-52e7d728-5521-4db3-9c37-e407c96a187c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:21.333769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 447/906] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:21.638079 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b5857-6fe2-4982-b08a-df8ee9360d61 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:21.695569 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:21.710248 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee834c5c-8c0b-4068-a81c-53a70cf3a0c8 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:21.712044 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee834c5c-8c0b-4068-a81c-53a70cf3a0c8 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:21.712815 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee834c5c-8c0b-4068-a81c-53a70cf3a0c8 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:21.723143 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 448/907] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:21 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:10:21.727709 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e1a82-2fcc-4e45-a0d8-e4cc11883660 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:21.729481 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e1a82-2fcc-4e45-a0d8-e4cc11883660 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:21.730397 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e1a82-2fcc-4e45-a0d8-e4cc11883660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:21.765338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-594e1a82-2fcc-4e45-a0d8-e4cc11883660 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:21.767543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-594e1a82-2fcc-4e45-a0d8-e4cc11883660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:21.835805 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b5857-6fe2-4982-b08a-df8ee9360d61 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:21.876930 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:21.906141 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efd647-e557-472c-91b0-28bee42df7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:21.907165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efd647-e557-472c-91b0-28bee42df7b2 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:21.907468 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efd647-e557-472c-91b0-28bee42df7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:21.907749 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efd647-e557-472c-91b0-28bee42df7b2 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:21.908552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72efd647-e557-472c-91b0-28bee42df7b2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:21.910203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72efd647-e557-472c-91b0-28bee42df7b2 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:21.917509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 449/908] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:10:21.918878 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e1a82-2fcc-4e45-a0d8-e4cc11883660 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:22.016721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 460/909] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 7 13:10:21 2022] POST /identity/v3/services => generated 241 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:10:22.107429 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e1a82-2fcc-4e45-a0d8-e4cc11883660 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:22.187239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c2404-7d1c-4fdc-ad82-58a93e6f2341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:22.188494 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c2404-7d1c-4fdc-ad82-58a93e6f2341 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:22.189002 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c2404-7d1c-4fdc-ad82-58a93e6f2341 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:22.189477 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c2404-7d1c-4fdc-ad82-58a93e6f2341 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:22.190604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c2404-7d1c-4fdc-ad82-58a93e6f2341 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:22.193044 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c2404-7d1c-4fdc-ad82-58a93e6f2341 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:22.212043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-779c2404-7d1c-4fdc-ad82-58a93e6f2341 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:10:22.212043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:22.212043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:22.212043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:22.212043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:22.212043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:22.212043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:22.212043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:22.212043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:10:22.212043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:22.212043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:22.212043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:22.212043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:22.212043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:22.212043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:22.212043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:22.213857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:22.213857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:22.213857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:22.213857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:22.213857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:22.213857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:22.213857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:22.213857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:22.213857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:22.213857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:22.213857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:22.213857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:22.213857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:22.213857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:22.213857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:22.213857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:22.213857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:22.216704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:22.216704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:22.216704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:22.216704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:22.216704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:22.216704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:22.216704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:22.216704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:22.216704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:22.216704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:22.216704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:22.216704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:22.216704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:10:22.216704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:22.216704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 450/910] 10.222.0.62 () {56 vars in 1132 bytes} [Wed Sep 7 13:10:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:22.225437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efd647-e557-472c-91b0-28bee42df7b2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:22.382361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c2404-7d1c-4fdc-ad82-58a93e6f2341 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:22.413028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:22.421967 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efd647-e557-472c-91b0-28bee42df7b2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:22.465123 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:22.471797 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0132e3-2a09-44a4-97d2-cb2ac95fe1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:22.472560 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0132e3-2a09-44a4-97d2-cb2ac95fe1fa None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:22.472847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0132e3-2a09-44a4-97d2-cb2ac95fe1fa None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:22.473132 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0132e3-2a09-44a4-97d2-cb2ac95fe1fa None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:22.473874 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0132e3-2a09-44a4-97d2-cb2ac95fe1fa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:22.474185 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0132e3-2a09-44a4-97d2-cb2ac95fe1fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:22.479060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0132e3-2a09-44a4-97d2-cb2ac95fe1fa None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:22.494009 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 461/911] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 7 13:10:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:22.499635 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0132e3-2a09-44a4-97d2-cb2ac95fe1fa None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:22.591936 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c2404-7d1c-4fdc-ad82-58a93e6f2341 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:22.646583 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3905705-0ea6-489b-8408-798a94a27f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:22.647759 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3905705-0ea6-489b-8408-798a94a27f15 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:22.648201 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3905705-0ea6-489b-8408-798a94a27f15 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:22.648670 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3905705-0ea6-489b-8408-798a94a27f15 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:22.649487 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3905705-0ea6-489b-8408-798a94a27f15 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:22.651291 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3905705-0ea6-489b-8408-798a94a27f15 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:22.662999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-d3905705-0ea6-489b-8408-798a94a27f15 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:10:22.662999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:22.662999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:22.662999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:22.662999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:22.662999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:22.662999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:22.662999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:22.662999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:10:22.662999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:22.662999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:22.662999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:22.662999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:22.662999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:22.662999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:22.662999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:22.662999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:22.664100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:22.664100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:22.664100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:22.664100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:22.664100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:22.664100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:22.664100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:22.664100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:22.664100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:22.664100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:22.664100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:22.664100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:22.664100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:22.664100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:22.664100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:22.664100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:22.664952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:22.664952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:22.664952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:22.664952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:22.664952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:22.664952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:22.664952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:22.664952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:22.664952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:22.664952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:22.664952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:22.664952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:22.664952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 13:10:22.664952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:22.668085 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 451/912] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:10:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:22.679275 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3905705-0ea6-489b-8408-798a94a27f15 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:22.698424 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0132e3-2a09-44a4-97d2-cb2ac95fe1fa None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:22.768722 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01a456-bc29-4ae2-8d90-3fa58c0cd2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:22.770139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01a456-bc29-4ae2-8d90-3fa58c0cd2b4 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:22.770767 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01a456-bc29-4ae2-8d90-3fa58c0cd2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:22.772282 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01a456-bc29-4ae2-8d90-3fa58c0cd2b4 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:22.774615 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01a456-bc29-4ae2-8d90-3fa58c0cd2b4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:22.775725 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01a456-bc29-4ae2-8d90-3fa58c0cd2b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:22.780493 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01a456-bc29-4ae2-8d90-3fa58c0cd2b4 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:22.792878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 462/913] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 7 13:10:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:22.802475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01a456-bc29-4ae2-8d90-3fa58c0cd2b4 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:22.861738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:22.896122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3905705-0ea6-489b-8408-798a94a27f15 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:22.901951 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:22.979336 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01b1b2-80ac-4fb3-b24d-b0f4e4a37ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:22.980000 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01b1b2-80ac-4fb3-b24d-b0f4e4a37ba2 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:22.980279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01b1b2-80ac-4fb3-b24d-b0f4e4a37ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:22.980544 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01b1b2-80ac-4fb3-b24d-b0f4e4a37ba2 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:22.981284 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01b1b2-80ac-4fb3-b24d-b0f4e4a37ba2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:22.981700 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01b1b2-80ac-4fb3-b24d-b0f4e4a37ba2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:22.983978 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01b1b2-80ac-4fb3-b24d-b0f4e4a37ba2 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:22.992119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 452/914] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:10:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:22.997695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01b1b2-80ac-4fb3-b24d-b0f4e4a37ba2 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:23.010808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01a456-bc29-4ae2-8d90-3fa58c0cd2b4 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:23.067486 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b1736-2296-44f9-a806-bd066b38ecbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:23.068687 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b1736-2296-44f9-a806-bd066b38ecbb None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:23.069292 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b1736-2296-44f9-a806-bd066b38ecbb None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:23.070105 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b1736-2296-44f9-a806-bd066b38ecbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:23.072159 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b1736-2296-44f9-a806-bd066b38ecbb None admin] RBAC: Adding query filter params (interface=public, service_id=2f42d7d22720496e9b0effef8cdc42cf, region_id=RegionOne) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:23.073061 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b1736-2296-44f9-a806-bd066b38ecbb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:23.075964 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b1736-2296-44f9-a806-bd066b38ecbb None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:23.088904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 463/915] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Sep 7 13:10:22 2022] GET /identity/v3/endpoints?service_id=2f42d7d22720496e9b0effef8cdc42cf&interface=public®ion_id=RegionOne => generated 194 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:23.094572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b1736-2296-44f9-a806-bd066b38ecbb None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:23.157344 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01b1b2-80ac-4fb3-b24d-b0f4e4a37ba2 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:23.244560 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665c8d2-20c9-4798-b419-47881ad246f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:23.245634 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c8d2-20c9-4798-b419-47881ad246f9 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:23.246118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c8d2-20c9-4798-b419-47881ad246f9 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:23.246609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665c8d2-20c9-4798-b419-47881ad246f9 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:23.247915 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665c8d2-20c9-4798-b419-47881ad246f9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:23.248307 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665c8d2-20c9-4798-b419-47881ad246f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:23.250812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665c8d2-20c9-4798-b419-47881ad246f9 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:23.262867 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 453/916] 10.222.0.62 () {56 vars in 1114 bytes} [Wed Sep 7 13:10:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:23.270440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665c8d2-20c9-4798-b419-47881ad246f9 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:23.278832 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:23.316571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b1736-2296-44f9-a806-bd066b38ecbb None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:23.328707 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:23.398656 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8504a4-b238-4c21-9c8a-f4b91946f22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:23.400513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8504a4-b238-4c21-9c8a-f4b91946f22c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:23.401294 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8504a4-b238-4c21-9c8a-f4b91946f22c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:23.402024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8504a4-b238-4c21-9c8a-f4b91946f22c None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:23.402961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8504a4-b238-4c21-9c8a-f4b91946f22c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:23.406728 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8504a4-b238-4c21-9c8a-f4b91946f22c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:23.420223 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 464/917] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 7 13:10:23 2022] GET /identity/v3/services => generated 1497 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:10:23.492291 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665c8d2-20c9-4798-b419-47881ad246f9 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:23.543868 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732c81d-3bf1-4d3c-b2c7-38d77f175c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:23.544578 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732c81d-3bf1-4d3c-b2c7-38d77f175c6d None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:23.544865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732c81d-3bf1-4d3c-b2c7-38d77f175c6d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:23.545145 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732c81d-3bf1-4d3c-b2c7-38d77f175c6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:23.545883 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732c81d-3bf1-4d3c-b2c7-38d77f175c6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:23.548347 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732c81d-3bf1-4d3c-b2c7-38d77f175c6d None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:23.679274 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 454/918] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:10:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:10:24.296643 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794297d-128a-4f38-8883-daa32e2f6b2f None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:24.297624 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794297d-128a-4f38-8883-daa32e2f6b2f None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:24.297977 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794297d-128a-4f38-8883-daa32e2f6b2f None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:24.299818 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 465/919] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:24.306097 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe316d-7bc0-4f4c-af42-f33c432b429f None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:24.306867 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe316d-7bc0-4f4c-af42-f33c432b429f None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:24.306867 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe316d-7bc0-4f4c-af42-f33c432b429f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:24.330494 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-95fe316d-7bc0-4f4c-af42-f33c432b429f None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:24.339923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-95fe316d-7bc0-4f4c-af42-f33c432b429f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:24.473249 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 455/920] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:24.867160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794297d-128a-4f38-8883-daa32e2f6b2f None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:25.027232 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794297d-128a-4f38-8883-daa32e2f6b2f None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:25.075935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e07e4-bf0a-4447-b951-9be295c1cd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:25.076596 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e07e4-bf0a-4447-b951-9be295c1cd0d None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:25.077596 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e07e4-bf0a-4447-b951-9be295c1cd0d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:25.078346 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e07e4-bf0a-4447-b951-9be295c1cd0d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:25.091410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e07e4-bf0a-4447-b951-9be295c1cd0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:10:25.097504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-9c1e07e4-bf0a-4447-b951-9be295c1cd0d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:10:25.097504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:25.097504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:25.097504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:25.097504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:25.097504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:25.097504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:25.097504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:25.097504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:25.097504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:25.097504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:25.097504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:25.097504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:10:25.097504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:10:25.097504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:10:25.097504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:10:25.098771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:10:25.098771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:10:25.098771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:10:25.098771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:10:25.098771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:25.098771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:25.098771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:25.098771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:25.098771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:25.098771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:25.098771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:25.098771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:25.098771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:25.098771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:25.098771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:25.098771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:25.099720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:25.099720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:25.099720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:25.099720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:25.099720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:10:25.099720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:10:25.099720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:10:25.099720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:10:25.099720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:10:25.099720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:10:25.099720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:10:25.099720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:25.099720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 466/921] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Sep 7 13:10:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:25.106456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe316d-7bc0-4f4c-af42-f33c432b429f None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:25.134632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:25.207671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2cf1c-7da9-4ad6-980d-2442f53289a6 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:25.208168 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2cf1c-7da9-4ad6-980d-2442f53289a6 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:25.208663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2cf1c-7da9-4ad6-980d-2442f53289a6 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:25.215543 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 467/922] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:25.221272 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323defc0-6a87-4367-8036-ddc00392b4a0 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:25.221892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323defc0-6a87-4367-8036-ddc00392b4a0 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:25.222077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323defc0-6a87-4367-8036-ddc00392b4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:25.256869 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-323defc0-6a87-4367-8036-ddc00392b4a0 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:25.258759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-323defc0-6a87-4367-8036-ddc00392b4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:25.265812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:25.291769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe316d-7bc0-4f4c-af42-f33c432b429f None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:25.350742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d6c84e-551e-4a99-80d5-72976c549fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:25.352024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6c84e-551e-4a99-80d5-72976c549fde None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:25.352635 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6c84e-551e-4a99-80d5-72976c549fde None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:25.353240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6c84e-551e-4a99-80d5-72976c549fde None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:25.354739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d6c84e-551e-4a99-80d5-72976c549fde None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:25.355404 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d6c84e-551e-4a99-80d5-72976c549fde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:25.358227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d6c84e-551e-4a99-80d5-72976c549fde None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:25.383165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 456/923] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Sep 7 13:10:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:25.385630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d6c84e-551e-4a99-80d5-72976c549fde None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:25.407173 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 468/924] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:25.563646 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:25.605193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d6c84e-551e-4a99-80d5-72976c549fde None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:25.675323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c966cc-5660-453d-bb6d-f921b459ea63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:25.676987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c966cc-5660-453d-bb6d-f921b459ea63 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:25.676987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c966cc-5660-453d-bb6d-f921b459ea63 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:25.677767 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c966cc-5660-453d-bb6d-f921b459ea63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:25.698994 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c966cc-5660-453d-bb6d-f921b459ea63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:25.702478 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c966cc-5660-453d-bb6d-f921b459ea63 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:25.732267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-47c966cc-5660-453d-bb6d-f921b459ea63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:25.732267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:25.732267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:25.732267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:25.732267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:25.732267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:25.732267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:25.732267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:25.732267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:25.732267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:25.732267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:25.732267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:25.732267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:25.732267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:25.732267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:25.732267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:25.734154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:25.734154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:25.734154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:25.734154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:25.734154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:25.734154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:25.734154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:25.734154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:25.734154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:25.734154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:25.734154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:25.734154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:25.734154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:25.734154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:25.734154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:25.734154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:25.734154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:25.736460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:25.736460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:25.736460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:25.739168 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 457/925] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:25.744153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323defc0-6a87-4367-8036-ddc00392b4a0 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:25.941986 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c966cc-5660-453d-bb6d-f921b459ea63 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:25.944018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323defc0-6a87-4367-8036-ddc00392b4a0 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:25.999731 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cacf86-dcd4-470f-8848-5478a21adcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:26.001401 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cacf86-dcd4-470f-8848-5478a21adcd2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:26.002179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cacf86-dcd4-470f-8848-5478a21adcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:26.002926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cacf86-dcd4-470f-8848-5478a21adcd2 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:26.004625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cacf86-dcd4-470f-8848-5478a21adcd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:26.005440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cacf86-dcd4-470f-8848-5478a21adcd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:26.008316 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cacf86-dcd4-470f-8848-5478a21adcd2 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:26.026241 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:26.034540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 469/926] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:26.045274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cacf86-dcd4-470f-8848-5478a21adcd2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:26.111085 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c966cc-5660-453d-bb6d-f921b459ea63 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:26.162302 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:26.176020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df714d6-69c0-48e5-8a2a-5efe34467cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:26.177216 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df714d6-69c0-48e5-8a2a-5efe34467cf1 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:26.177799 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df714d6-69c0-48e5-8a2a-5efe34467cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:26.178455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df714d6-69c0-48e5-8a2a-5efe34467cf1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:26.179620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df714d6-69c0-48e5-8a2a-5efe34467cf1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:26.182032 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df714d6-69c0-48e5-8a2a-5efe34467cf1 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:26.199502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-5df714d6-69c0-48e5-8a2a-5efe34467cf1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:10:26.199502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:26.199502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:26.199502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:26.199502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:26.199502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:26.199502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:26.199502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:26.199502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:10:26.199502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:26.199502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:26.199502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:26.199502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:26.199502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:26.199502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:26.199502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:26.199502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:26.200846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:26.200846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:26.200846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:26.200846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:26.200846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:26.200846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:26.200846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:26.200846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:26.200846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:26.200846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:26.200846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:26.200846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:26.200846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:26.200846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:26.200846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:26.200846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:26.202725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:26.202725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:26.202725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:26.202725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:26.202725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:26.202725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:26.202725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:26.202725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:26.202725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:26.202725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:26.202725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:26.202725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:26.202725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:10:26.202725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:26.211162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 458/927] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 7 13:10:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:26.215567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df714d6-69c0-48e5-8a2a-5efe34467cf1 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:26.254465 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cacf86-dcd4-470f-8848-5478a21adcd2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:26.329711 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5315122-96e0-47fb-bc69-0739f023886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:26.330698 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5315122-96e0-47fb-bc69-0739f023886e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:26.331089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5315122-96e0-47fb-bc69-0739f023886e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:26.336848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5315122-96e0-47fb-bc69-0739f023886e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:26.346312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5315122-96e0-47fb-bc69-0739f023886e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:26.348756 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5315122-96e0-47fb-bc69-0739f023886e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:26.369219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-a5315122-96e0-47fb-bc69-0739f023886e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:10:26.369219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:26.369219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:26.369219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:26.369219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:26.369219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:26.369219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:26.369219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:26.369219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:26.369219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:26.369219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:26.369219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:26.369219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:10:26.369219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:10:26.369219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:10:26.369219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:26.372726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:26.383930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:26.383930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:26.383930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:10:26.383930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:10:26.383930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:10:26.383930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:10:26.383930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:10:26.383930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:10:26.383930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:10:26.383930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:26.383930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 470/928] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 7 13:10:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:26.383930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5315122-96e0-47fb-bc69-0739f023886e None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:26.407366 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df714d6-69c0-48e5-8a2a-5efe34467cf1 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:26.482262 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:26.496931 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96fbfd-e1ca-458f-a924-e0a97dc1f666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:26.499507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96fbfd-e1ca-458f-a924-e0a97dc1f666 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:26.502462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96fbfd-e1ca-458f-a924-e0a97dc1f666 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:26.502462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96fbfd-e1ca-458f-a924-e0a97dc1f666 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:26.504142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96fbfd-e1ca-458f-a924-e0a97dc1f666 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:26.504986 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96fbfd-e1ca-458f-a924-e0a97dc1f666 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:26.508013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96fbfd-e1ca-458f-a924-e0a97dc1f666 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:26.523197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 459/929] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 7 13:10:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:26.536827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96fbfd-e1ca-458f-a924-e0a97dc1f666 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:26.558479 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5315122-96e0-47fb-bc69-0739f023886e None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:26.632922 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b666d6-89e3-429f-808c-882c80b0825a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:26.634647 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b666d6-89e3-429f-808c-882c80b0825a None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:26.635472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b666d6-89e3-429f-808c-882c80b0825a None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:26.636246 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b666d6-89e3-429f-808c-882c80b0825a None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:26.638070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b666d6-89e3-429f-808c-882c80b0825a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:26.639219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b666d6-89e3-429f-808c-882c80b0825a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:26.642903 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b666d6-89e3-429f-808c-882c80b0825a None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:26.659512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:26.686348 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 471/930] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Sep 7 13:10:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:26.695666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b666d6-89e3-429f-808c-882c80b0825a None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:26.734384 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96fbfd-e1ca-458f-a924-e0a97dc1f666 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:26.816103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1f541-e071-453e-8b18-9298d3ea60dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:26.819399 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1f541-e071-453e-8b18-9298d3ea60dc None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:26.820747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1f541-e071-453e-8b18-9298d3ea60dc None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:26.821592 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1f541-e071-453e-8b18-9298d3ea60dc None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:26.823510 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1f541-e071-453e-8b18-9298d3ea60dc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:26.824421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1f541-e071-453e-8b18-9298d3ea60dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:26.829036 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1f541-e071-453e-8b18-9298d3ea60dc None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:26.843064 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 460/931] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 7 13:10:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:26.853560 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1f541-e071-453e-8b18-9298d3ea60dc None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:26.944033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b666d6-89e3-429f-808c-882c80b0825a None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:26.982025 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:27.010008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d139076-c28c-432b-a8d9-4eb41c307fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:27.011031 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d139076-c28c-432b-a8d9-4eb41c307fb4 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:27.011556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d139076-c28c-432b-a8d9-4eb41c307fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:27.018444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d139076-c28c-432b-a8d9-4eb41c307fb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:27.033503 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d139076-c28c-432b-a8d9-4eb41c307fb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:27.036513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d139076-c28c-432b-a8d9-4eb41c307fb4 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:27.047408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1f541-e071-453e-8b18-9298d3ea60dc None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:27.064665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-7d139076-c28c-432b-a8d9-4eb41c307fb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:27.064665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:27.064665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:27.064665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:27.064665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:27.064665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:27.064665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:27.064665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:27.064665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:27.064665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:27.064665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:27.064665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:27.064665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:27.064665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:27.064665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:27.064665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:27.075193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:27.075193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:27.075193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:27.075193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:27.075193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:27.075193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:27.075193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:27.075193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:27.075193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:27.075193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:27.075193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:27.075193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:27.075193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:27.075193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:27.075193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:27.075193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:27.075193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:27.078190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:27.078190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:27.078190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:27.078190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 472/932] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:27.078190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a87ffb-0a66-41de-b8f9-a59dc89f8110 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:27.078190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a87ffb-0a66-41de-b8f9-a59dc89f8110 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:27.078190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a87ffb-0a66-41de-b8f9-a59dc89f8110 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:27.078190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 473/933] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:27.080471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae9886-a452-4d65-b10a-45a65dc1e0df None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:27.080881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae9886-a452-4d65-b10a-45a65dc1e0df None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:27.081300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae9886-a452-4d65-b10a-45a65dc1e0df None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:27.083607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 474/934] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:27.084327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae9886-a452-4d65-b10a-45a65dc1e0df None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:27.086119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:27.114291 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01cb6ba-38da-43bb-94f1-f17a50a51c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:27.115444 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cb6ba-38da-43bb-94f1-f17a50a51c31 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:27.115881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cb6ba-38da-43bb-94f1-f17a50a51c31 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:27.116322 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cb6ba-38da-43bb-94f1-f17a50a51c31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:27.117635 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01cb6ba-38da-43bb-94f1-f17a50a51c31 None admin] RBAC: Adding query filter params (interface=public, service_id=f09022ea4c56493fbc8c07ba240f0243, region_id=RegionOne) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:27.118118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01cb6ba-38da-43bb-94f1-f17a50a51c31 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:27.120527 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01cb6ba-38da-43bb-94f1-f17a50a51c31 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:27.131205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 461/935] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Sep 7 13:10:26 2022] GET /identity/v3/endpoints?service_id=f09022ea4c56493fbc8c07ba240f0243&interface=public®ion_id=RegionOne => generated 194 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:27.133107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545671c0-139c-4f05-a47f-e1d95c9ede22 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:27.133548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545671c0-139c-4f05-a47f-e1d95c9ede22 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:27.133981 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545671c0-139c-4f05-a47f-e1d95c9ede22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:27.165132 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-545671c0-139c-4f05-a47f-e1d95c9ede22 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:27.167196 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-545671c0-139c-4f05-a47f-e1d95c9ede22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:27.293050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae9886-a452-4d65-b10a-45a65dc1e0df None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:27.294103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 462/936] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:27.296909 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf9306-9166-4d63-bb35-c7b74ee99085 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:27.297278 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf9306-9166-4d63-bb35-c7b74ee99085 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:27.297605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf9306-9166-4d63-bb35-c7b74ee99085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:27.323933 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-d6cf9306-9166-4d63-bb35-c7b74ee99085 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:27.325258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d6cf9306-9166-4d63-bb35-c7b74ee99085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:27.359828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df52e45-2620-4f12-aa00-43cb2914e8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:27.360978 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df52e45-2620-4f12-aa00-43cb2914e8c2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:27.361410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df52e45-2620-4f12-aa00-43cb2914e8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:27.361814 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df52e45-2620-4f12-aa00-43cb2914e8c2 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:27.362628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df52e45-2620-4f12-aa00-43cb2914e8c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:27.363170 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df52e45-2620-4f12-aa00-43cb2914e8c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:27.364985 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df52e45-2620-4f12-aa00-43cb2914e8c2 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:27.396247 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 475/937] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:27.396247 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df52e45-2620-4f12-aa00-43cb2914e8c2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:27.447337 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 463/938] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:27.447337 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf9306-9166-4d63-bb35-c7b74ee99085 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:27.491585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:27.542773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:27.647699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df52e45-2620-4f12-aa00-43cb2914e8c2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:27.657835 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf9306-9166-4d63-bb35-c7b74ee99085 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:27.696294 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7e3d2-a239-4fb9-881c-08f5e501a344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:27.696953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7e3d2-a239-4fb9-881c-08f5e501a344 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:27.697235 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7e3d2-a239-4fb9-881c-08f5e501a344 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:27.697681 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7e3d2-a239-4fb9-881c-08f5e501a344 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:27.698345 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7e3d2-a239-4fb9-881c-08f5e501a344 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:27.699841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7e3d2-a239-4fb9-881c-08f5e501a344 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:27.708033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 476/939] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 7 13:10:27 2022] GET /identity/v3/services => generated 1497 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:10:27.708913 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7e3d2-a239-4fb9-881c-08f5e501a344 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:27.737326 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099dd5e-5739-4a50-ad26-971bd27ddef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:27.738387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099dd5e-5739-4a50-ad26-971bd27ddef2 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:27.738872 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099dd5e-5739-4a50-ad26-971bd27ddef2 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:27.739786 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099dd5e-5739-4a50-ad26-971bd27ddef2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:27.755378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099dd5e-5739-4a50-ad26-971bd27ddef2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:27.759286 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099dd5e-5739-4a50-ad26-971bd27ddef2 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:27.782765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-5099dd5e-5739-4a50-ad26-971bd27ddef2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:27.782765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:27.782765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:27.782765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:27.782765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:27.782765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:27.782765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:27.782765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:27.782765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:27.782765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:27.782765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:27.782765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:27.782765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:10:27.782765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:10:27.782765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:10:27.785124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:10:27.785124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:27.785124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:27.785124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:27.785124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:27.785124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:27.785124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:27.785124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:27.785124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:27.785124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:27.785124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:27.785124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:27.785124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:27.785124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:27.785124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:27.785124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:27.785124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:27.786766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:10:27.786766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:10:27.786766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:10:27.786766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:10:27.786766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:10:27.786766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:10:27.786766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:27.786766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:27.786766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 464/940] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:10:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:27.786766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099dd5e-5739-4a50-ad26-971bd27ddef2 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:27.883260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:27.897557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7e3d2-a239-4fb9-881c-08f5e501a344 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:27.970579 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099dd5e-5739-4a50-ad26-971bd27ddef2 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:27.973621 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d85899-f453-4295-93e4-b0dc07af8842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:27.974313 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d85899-f453-4295-93e4-b0dc07af8842 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:27.974552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:27.974719 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d85899-f453-4295-93e4-b0dc07af8842 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:27.974823 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d85899-f453-4295-93e4-b0dc07af8842 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:27.975517 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d85899-f453-4295-93e4-b0dc07af8842 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:27.976997 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d85899-f453-4295-93e4-b0dc07af8842 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:27.990952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-e9d85899-f453-4295-93e4-b0dc07af8842 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:10:27.990952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:27.990952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:27.990952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:27.990952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:27.990952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:27.990952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:27.990952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:27.990952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:10:27.990952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:27.990952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:27.990952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:27.990952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:27.990952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:27.990952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:27.990952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:27.995848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:27.995848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:27.995848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:27.995848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:27.995848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:27.995848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:27.995848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:27.995848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:27.995848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:27.995848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:27.995848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:27.995848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:27.995848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:27.995848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:27.995848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:27.995848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:27.995848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:27.997784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:27.997784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:27.997784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:27.997784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:27.997784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:27.997784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:27.997784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:27.997784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:27.997784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:27.997784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:27.997784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:27.997784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:27.997784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 13:10:27.997784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:27.997784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 477/941] 10.222.0.62 () {56 vars in 1132 bytes} [Wed Sep 7 13:10:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:28.001548 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d85899-f453-4295-93e4-b0dc07af8842 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:28.044107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f8400-95b4-4b08-863f-b8aa3234e629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:28.045064 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f8400-95b4-4b08-863f-b8aa3234e629 None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:28.045348 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f8400-95b4-4b08-863f-b8aa3234e629 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:28.046275 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f8400-95b4-4b08-863f-b8aa3234e629 None admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:28.047561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f8400-95b4-4b08-863f-b8aa3234e629 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:28.050275 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f8400-95b4-4b08-863f-b8aa3234e629 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:28.054921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: WARNING keystone.server.flask.common [None req-a07f8400-95b4-4b08-863f-b8aa3234e629 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 07 13:10:28.111403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.sql.core [None req-a07f8400-95b4-4b08-863f-b8aa3234e629 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 13:10:28.111403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 07 13:10:28.111403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [parameters: {'id': '07de6afff0d5482a8cd6c3f0e2e8dd53', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 07 13:10:28.111403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=105316) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 13:10:28.143245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-a07f8400-95b4-4b08-863f-b8aa3234e629 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 07de6afff0d5482a8cd6c3f0e2e8dd53.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 07de6afff0d5482a8cd6c3f0e2e8dd53. Sep 07 13:10:28.143245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:28.143245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 13:10:28.143245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 13:10:28.143245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 13:10:28.143245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 13:10:28.143245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 13:10:28.143245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 13:10:28.143245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 13:10:28.143245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application conn.query(q) Sep 07 13:10:28.143245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 13:10:28.143245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 13:10:28.143245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application result.read() Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 07 13:10:28.144473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 07 13:10:28.145927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 13:10:28.145927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application next(self.gen) Sep 07 13:10:28.145927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 13:10:28.145927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application yield resource Sep 07 13:10:28.145927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 13:10:28.145927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application next(self.gen) Sep 07 13:10:28.145927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 13:10:28.145927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 13:10:28.145927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 13:10:28.145927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 13:10:28.145927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 13:10:28.145927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise self.value Sep 07 13:10:28.145927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 13:10:28.145927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 13:10:28.145927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 13:10:28.145927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application session.commit() Sep 07 13:10:28.145927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 13:10:28.146859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 13:10:28.146859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 13:10:28.146859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 13:10:28.146859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 13:10:28.146859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self.session.flush() Sep 07 13:10:28.146859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 13:10:28.146859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 13:10:28.146859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 13:10:28.146859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 13:10:28.146859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 13:10:28.146859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application compat.raise_( Sep 07 13:10:28.146859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 13:10:28.146859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception Sep 07 13:10:28.146859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 13:10:28.146859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 13:10:28.146859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 13:10:28.146859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 13:10:28.147886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 13:10:28.147886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 13:10:28.147886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 13:10:28.147886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 13:10:28.147886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 07 13:10:28.147886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 07 13:10:28.147886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 13:10:28.147886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 13:10:28.147886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 13:10:28.147886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 13:10:28.147886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 13:10:28.147886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 13:10:28.147886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 13:10:28.147886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 13:10:28.147886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 13:10:28.148962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 13:10:28.148962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 13:10:28.148962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception Sep 07 13:10:28.148962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 13:10:28.148962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 13:10:28.148962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 13:10:28.148962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 13:10:28.148962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 13:10:28.148962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 13:10:28.148962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 13:10:28.148962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application conn.query(q) Sep 07 13:10:28.148962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 13:10:28.148962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 13:10:28.148962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 13:10:28.148962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application result.read() Sep 07 13:10:28.148962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 13:10:28.148962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 13:10:28.150055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 13:10:28.150055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 13:10:28.150055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 13:10:28.150055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 13:10:28.150055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 13:10:28.150055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 13:10:28.150055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 13:10:28.150055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 07 13:10:28.150055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [parameters: {'id': '07de6afff0d5482a8cd6c3f0e2e8dd53', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 07 13:10:28.150055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 13:10:28.150055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:28.150055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:28.150055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:28.150055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:28.150055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 07 13:10:28.150957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 07 13:10:28.150957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 13:10:28.150957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 13:10:28.150957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 07 13:10:28.150957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:28.150957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:28.150957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:28.150957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:28.150957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:28.150957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:28.150957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:28.150957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:28.150957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:28.150957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:28.150957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:28.150957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:28.150957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:28.152383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:28.152383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 07 13:10:28.152383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 07 13:10:28.152383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:28.152383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:28.152383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 07 13:10:28.152383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 07 13:10:28.152383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 07de6afff0d5482a8cd6c3f0e2e8dd53. Sep 07 13:10:28.152383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:28.153817 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 465/942] 10.222.0.62 () {60 vars in 1140 bytes} [Wed Sep 7 13:10:27 2022] POST /identity/v3/projects => generated 262 bytes in 369 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:28.156440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f8400-95b4-4b08-863f-b8aa3234e629 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:28.180306 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d85899-f453-4295-93e4-b0dc07af8842 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:28.238549 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636ea3e-1142-4f60-b634-159004022374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:28.239329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636ea3e-1142-4f60-b634-159004022374 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:28.239638 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636ea3e-1142-4f60-b634-159004022374 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:28.240006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636ea3e-1142-4f60-b634-159004022374 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:28.240796 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636ea3e-1142-4f60-b634-159004022374 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:28.241112 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636ea3e-1142-4f60-b634-159004022374 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:28.242733 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636ea3e-1142-4f60-b634-159004022374 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:28.269242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 478/943] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:10:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:28.269242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636ea3e-1142-4f60-b634-159004022374 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:28.295441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:28.344010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f8400-95b4-4b08-863f-b8aa3234e629 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:28.392575 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e310d-c994-4fc5-8672-e41727583735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:28.393487 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e310d-c994-4fc5-8672-e41727583735 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:28.393980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e310d-c994-4fc5-8672-e41727583735 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:28.394399 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e310d-c994-4fc5-8672-e41727583735 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:28.395274 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e310d-c994-4fc5-8672-e41727583735 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:28.395734 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e310d-c994-4fc5-8672-e41727583735 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:28.397558 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e310d-c994-4fc5-8672-e41727583735 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:28.412438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 466/944] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 7 13:10:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:28.412438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e310d-c994-4fc5-8672-e41727583735 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:28.445412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636ea3e-1142-4f60-b634-159004022374 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:28.451515 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:28.504708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805b5ca-71e9-4549-85b0-78305638ede6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:28.505692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805b5ca-71e9-4549-85b0-78305638ede6 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:28.505939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805b5ca-71e9-4549-85b0-78305638ede6 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:28.506213 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805b5ca-71e9-4549-85b0-78305638ede6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:28.515335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805b5ca-71e9-4549-85b0-78305638ede6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:28.517184 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805b5ca-71e9-4549-85b0-78305638ede6 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:28.532364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-6805b5ca-71e9-4549-85b0-78305638ede6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:28.532364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:28.532364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:28.532364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:28.532364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:28.532364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:28.532364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:28.532364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:28.532364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:28.532364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:28.532364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:28.532364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:28.532364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:10:28.532364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:10:28.532364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:10:28.533557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:10:28.533557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:28.533557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:28.533557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:28.533557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:28.533557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:28.533557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:28.533557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:28.533557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:28.533557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:28.533557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:28.533557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:28.533557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:28.533557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:28.533557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:28.533557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:28.533557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:28.534826 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:10:28.534826 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:10:28.534826 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:10:28.534826 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:10:28.534826 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:10:28.534826 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:10:28.534826 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:28.534826 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:28.538877 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 479/945] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:10:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:28.539999 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805b5ca-71e9-4549-85b0-78305638ede6 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:28.568587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e310d-c994-4fc5-8672-e41727583735 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:28.635944 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9583585-da5b-4506-aa8c-58514f3660c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:28.637011 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9583585-da5b-4506-aa8c-58514f3660c7 None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:28.637458 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9583585-da5b-4506-aa8c-58514f3660c7 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:28.637919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9583585-da5b-4506-aa8c-58514f3660c7 None admin] PATH_INFO: `/v3/projects/4574cf3ff270485faf60594019f7e05c/users/8121ba4cf2cf44d9a5a55975ea85928c/roles/e88e2f39fcb94166851f6db7b9f9dfcb` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:28.642565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9583585-da5b-4506-aa8c-58514f3660c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=4574cf3ff270485faf60594019f7e05c, user_id=8121ba4cf2cf44d9a5a55975ea85928c, role_id=e88e2f39fcb94166851f6db7b9f9dfcb)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:28.645122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9583585-da5b-4506-aa8c-58514f3660c7 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:28.681278 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:28.710533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805b5ca-71e9-4549-85b0-78305638ede6 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:28.725582 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 467/946] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:10:28 2022] PUT /identity/v3/projects/4574cf3ff270485faf60594019f7e05c/users/8121ba4cf2cf44d9a5a55975ea85928c/roles/e88e2f39fcb94166851f6db7b9f9dfcb => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:10:28.725582 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9583585-da5b-4506-aa8c-58514f3660c7 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:28.773889 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4779895-9820-45a1-88a9-033b0fc158bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:28.774628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4779895-9820-45a1-88a9-033b0fc158bf None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:28.774968 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4779895-9820-45a1-88a9-033b0fc158bf None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:28.775335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4779895-9820-45a1-88a9-033b0fc158bf None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:28.776157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4779895-9820-45a1-88a9-033b0fc158bf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:28.776625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4779895-9820-45a1-88a9-033b0fc158bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:28.778358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4779895-9820-45a1-88a9-033b0fc158bf None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:28.791600 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 480/947] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 7 13:10:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:28.793246 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4779895-9820-45a1-88a9-033b0fc158bf None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:28.846512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:28.867968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9583585-da5b-4506-aa8c-58514f3660c7 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:28.917576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ce7cc-fb09-4ce5-af1c-550c91516536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:28.918261 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ce7cc-fb09-4ce5-af1c-550c91516536 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:28.918550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ce7cc-fb09-4ce5-af1c-550c91516536 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:28.918838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ce7cc-fb09-4ce5-af1c-550c91516536 None admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:28.919744 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ce7cc-fb09-4ce5-af1c-550c91516536 None admin] RBAC: Adding query filter params (name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:28.920062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ce7cc-fb09-4ce5-af1c-550c91516536 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:28.922768 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ce7cc-fb09-4ce5-af1c-550c91516536 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:28.948221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 468/948] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 7 13:10:28 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:29.022015 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4779895-9820-45a1-88a9-033b0fc158bf None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:29.093443 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcbc7a-949b-49c5-8428-991da430fc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:29.095048 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcbc7a-949b-49c5-8428-991da430fc7d None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:29.095675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcbc7a-949b-49c5-8428-991da430fc7d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:29.096197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcbc7a-949b-49c5-8428-991da430fc7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:29.097243 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fcbc7a-949b-49c5-8428-991da430fc7d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:29.099234 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fcbc7a-949b-49c5-8428-991da430fc7d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:29.186189 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:29.196572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 481/949] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:10:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:10:31.232321 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39e609-2dc6-4ebe-9782-32a5c189db43 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:31.233233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39e609-2dc6-4ebe-9782-32a5c189db43 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:31.233233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39e609-2dc6-4ebe-9782-32a5c189db43 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:31.235621 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 469/950] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:31.241542 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7252d0e-3042-442e-84da-a1a16364c933 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:31.242354 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7252d0e-3042-442e-84da-a1a16364c933 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:31.242354 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7252d0e-3042-442e-84da-a1a16364c933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:31.265894 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-d7252d0e-3042-442e-84da-a1a16364c933 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:31.267172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d7252d0e-3042-442e-84da-a1a16364c933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:31.412919 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 482/951] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:10:31.541766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09f112-6660-4c79-88bd-0274629c7a99 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:31.542272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09f112-6660-4c79-88bd-0274629c7a99 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:31.542770 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09f112-6660-4c79-88bd-0274629c7a99 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:31.546270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 470/952] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:31.555818 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340265d-f890-40f6-ae6d-062efd24155b None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:31.556266 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340265d-f890-40f6-ae6d-062efd24155b None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:31.556699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340265d-f890-40f6-ae6d-062efd24155b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:31.589754 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b340265d-f890-40f6-ae6d-062efd24155b None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:31.591919 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b340265d-f890-40f6-ae6d-062efd24155b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:31.711013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 483/953] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:31.714838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09f112-6660-4c79-88bd-0274629c7a99 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:31.928447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09f112-6660-4c79-88bd-0274629c7a99 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:31.975230 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b21df-2d12-41b5-bb1e-40bf09cd68ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:31.975955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b21df-2d12-41b5-bb1e-40bf09cd68ec None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:31.976235 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b21df-2d12-41b5-bb1e-40bf09cd68ec None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:31.976512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b21df-2d12-41b5-bb1e-40bf09cd68ec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:31.990353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b21df-2d12-41b5-bb1e-40bf09cd68ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:10:31.995751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-0b6b21df-2d12-41b5-bb1e-40bf09cd68ec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:10:31.995751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:31.995751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:31.995751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:31.995751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:31.995751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:31.995751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:31.995751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:31.995751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:31.995751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:31.995751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:31.995751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:31.995751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:10:31.995751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:10:31.995751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:10:31.995751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:10:31.997586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:10:31.997586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:10:31.997586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:10:31.997586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:10:31.997586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:31.997586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:31.997586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:31.997586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:31.997586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:31.997586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:31.997586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:31.997586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:31.997586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:31.997586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:31.997586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:31.997586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:32.000954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:32.000954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:32.000954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:32.000954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:32.000954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:10:32.000954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:10:32.000954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:10:32.000954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:10:32.000954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:10:32.000954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:10:32.000954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:10:32.000954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:32.000954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 471/954] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Sep 7 13:10:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:32.007694 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b21df-2d12-41b5-bb1e-40bf09cd68ec None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:32.008204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340265d-f890-40f6-ae6d-062efd24155b None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:32.019243 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:32.161646 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b21df-2d12-41b5-bb1e-40bf09cd68ec None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:32.164395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:32.227629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a23897-e37f-4b04-88a8-7ce23199ebca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:32.228620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a23897-e37f-4b04-88a8-7ce23199ebca None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:32.229131 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a23897-e37f-4b04-88a8-7ce23199ebca None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:32.229557 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a23897-e37f-4b04-88a8-7ce23199ebca None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:32.230409 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a23897-e37f-4b04-88a8-7ce23199ebca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:32.231085 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a23897-e37f-4b04-88a8-7ce23199ebca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:32.233196 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a23897-e37f-4b04-88a8-7ce23199ebca None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:32.246352 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 472/955] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Sep 7 13:10:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:32.255863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a23897-e37f-4b04-88a8-7ce23199ebca None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:32.267898 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b340265d-f890-40f6-ae6d-062efd24155b None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:32.341997 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333eef27-31b1-45a4-80a4-dbff1bdfdca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:32.343338 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333eef27-31b1-45a4-80a4-dbff1bdfdca2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:32.343761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333eef27-31b1-45a4-80a4-dbff1bdfdca2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:32.344243 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333eef27-31b1-45a4-80a4-dbff1bdfdca2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:32.345395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333eef27-31b1-45a4-80a4-dbff1bdfdca2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:32.348193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333eef27-31b1-45a4-80a4-dbff1bdfdca2 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:32.359644 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:32.368699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-333eef27-31b1-45a4-80a4-dbff1bdfdca2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:10:32.368699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:32.368699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:32.368699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:32.368699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:32.368699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:32.368699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:32.368699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:32.368699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:10:32.368699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:32.368699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:32.368699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:32.368699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:32.368699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:32.368699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:32.368699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:32.368699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:32.376119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:32.376119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:32.376119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:32.376119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:32.376119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:32.376119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:32.376119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:32.376119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:32.376119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:32.376119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:32.376119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:32.376119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:32.376119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:32.376119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:32.376119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:32.376119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:32.376998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:32.376998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:32.376998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:32.376998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:32.376998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:32.376998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:32.376998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:32.376998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:32.376998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:32.376998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:32.376998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:32.376998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:32.376998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:10:32.376998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:32.376998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 484/956] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 7 13:10:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:32.380266 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333eef27-31b1-45a4-80a4-dbff1bdfdca2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:32.417530 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a23897-e37f-4b04-88a8-7ce23199ebca None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:32.473764 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04477b56-358f-4579-81c0-6dc8e204e4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:32.474545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04477b56-358f-4579-81c0-6dc8e204e4a8 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:32.474961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04477b56-358f-4579-81c0-6dc8e204e4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:32.475410 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04477b56-358f-4579-81c0-6dc8e204e4a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:32.489672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04477b56-358f-4579-81c0-6dc8e204e4a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:32.491295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04477b56-358f-4579-81c0-6dc8e204e4a8 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:32.510304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-04477b56-358f-4579-81c0-6dc8e204e4a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:32.510304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:32.510304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:32.510304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:32.510304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:32.510304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:32.510304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:32.510304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:32.510304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:32.510304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:32.510304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:32.510304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:32.510304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:32.510304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:32.510304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:32.510304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:32.512459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:32.512459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:32.512459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:32.512459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:32.512459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:32.512459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:32.512459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:32.512459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:32.512459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:32.512459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:32.512459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:32.512459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:32.512459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:32.512459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:32.512459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:32.512459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:32.512459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:32.515467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:32.515467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:32.515467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:32.515467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 473/957] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:32.519917 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04477b56-358f-4579-81c0-6dc8e204e4a8 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:32.578596 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333eef27-31b1-45a4-80a4-dbff1bdfdca2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:32.599459 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:32.623411 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc8c94-357e-472a-b1fb-8f5930e5bb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:32.624542 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc8c94-357e-472a-b1fb-8f5930e5bb8c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:32.625342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc8c94-357e-472a-b1fb-8f5930e5bb8c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:32.625525 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc8c94-357e-472a-b1fb-8f5930e5bb8c None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:32.626697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc8c94-357e-472a-b1fb-8f5930e5bb8c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:32.627259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc8c94-357e-472a-b1fb-8f5930e5bb8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:32.629926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc8c94-357e-472a-b1fb-8f5930e5bb8c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:32.637488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 485/958] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 7 13:10:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:32.639220 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9a741-6cae-4520-a422-bf88dbbd5eed None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:32.640271 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9a741-6cae-4520-a422-bf88dbbd5eed None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:32.640271 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9a741-6cae-4520-a422-bf88dbbd5eed None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:32.642889 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 486/959] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:32.646780 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf7fbf-cbeb-4718-bfa5-cdd646bae338 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:32.647333 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf7fbf-cbeb-4718-bfa5-cdd646bae338 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:32.647815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf7fbf-cbeb-4718-bfa5-cdd646bae338 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:32.650272 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 487/960] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:32.652782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd852e01-7c28-4e4f-b1bc-e0e0c2911a59 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:32.653272 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd852e01-7c28-4e4f-b1bc-e0e0c2911a59 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:32.653721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd852e01-7c28-4e4f-b1bc-e0e0c2911a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:32.677179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04477b56-358f-4579-81c0-6dc8e204e4a8 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:32.688072 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-fd852e01-7c28-4e4f-b1bc-e0e0c2911a59 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:32.690167 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-fd852e01-7c28-4e4f-b1bc-e0e0c2911a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:32.697614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:32.717703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e79b6-b3ab-462c-91c4-63a3f7e58a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:32.718460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e79b6-b3ab-462c-91c4-63a3f7e58a74 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:32.718781 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e79b6-b3ab-462c-91c4-63a3f7e58a74 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:32.719319 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e79b6-b3ab-462c-91c4-63a3f7e58a74 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:32.719710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e79b6-b3ab-462c-91c4-63a3f7e58a74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:32.720007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e79b6-b3ab-462c-91c4-63a3f7e58a74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:32.721610 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e79b6-b3ab-462c-91c4-63a3f7e58a74 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:32.737473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 474/961] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:32.738454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e79b6-b3ab-462c-91c4-63a3f7e58a74 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:32.808697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 488/962] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:32.810393 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3855d-2e12-494e-99b0-d9a821b79dff None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:32.810823 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3855d-2e12-494e-99b0-d9a821b79dff None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:32.811797 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c3855d-2e12-494e-99b0-d9a821b79dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:32.846361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-c6c3855d-2e12-494e-99b0-d9a821b79dff None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:32.848380 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c6c3855d-2e12-494e-99b0-d9a821b79dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:32.925593 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e79b6-b3ab-462c-91c4-63a3f7e58a74 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:32.971355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 489/963] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:32.972693 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c3855d-2e12-494e-99b0-d9a821b79dff None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:33.003440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcde87f-1bea-4534-82b0-8437f3e35082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:33.004072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcde87f-1bea-4534-82b0-8437f3e35082 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:33.004350 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcde87f-1bea-4534-82b0-8437f3e35082 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:33.004669 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcde87f-1bea-4534-82b0-8437f3e35082 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:33.005322 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcde87f-1bea-4534-82b0-8437f3e35082 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:33.005624 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcde87f-1bea-4534-82b0-8437f3e35082 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:33.007303 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcde87f-1bea-4534-82b0-8437f3e35082 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:33.018744 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 475/964] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 7 13:10:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:33.020007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcde87f-1bea-4534-82b0-8437f3e35082 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:33.048717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:33.075831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:33.154731 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c3855d-2e12-494e-99b0-d9a821b79dff None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:33.176585 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcde87f-1bea-4534-82b0-8437f3e35082 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:33.208958 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55364818-9fc8-415f-9c26-8ecf260da420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:33.209602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55364818-9fc8-415f-9c26-8ecf260da420 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:33.209882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55364818-9fc8-415f-9c26-8ecf260da420 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:33.210151 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55364818-9fc8-415f-9c26-8ecf260da420 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:33.220516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55364818-9fc8-415f-9c26-8ecf260da420 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:33.222124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55364818-9fc8-415f-9c26-8ecf260da420 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:33.224797 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f4856-a99f-44d6-ad7d-03ab856161b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:33.225755 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f4856-a99f-44d6-ad7d-03ab856161b9 None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:33.226149 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f4856-a99f-44d6-ad7d-03ab856161b9 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:33.226474 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f4856-a99f-44d6-ad7d-03ab856161b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:33.227246 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f4856-a99f-44d6-ad7d-03ab856161b9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:33.228701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f4856-a99f-44d6-ad7d-03ab856161b9 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:33.238006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-55364818-9fc8-415f-9c26-8ecf260da420 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:10:33.238006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:33.238006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:33.238006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:33.238006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:33.238006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:33.238006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:33.238006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:33.238006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:33.238006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:33.238006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:33.238006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:33.238006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:10:33.238006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:10:33.238006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:10:33.238006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:33.239332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:33.240519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:33.240519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:33.240519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:10:33.240519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:10:33.240519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:10:33.240519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:10:33.240519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:10:33.240519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:10:33.240519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 13:10:33.240519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:33.242635 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 490/965] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 7 13:10:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:33.244170 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55364818-9fc8-415f-9c26-8ecf260da420 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:33.335821 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 476/966] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:10:33 2022] POST /identity/v3/endpoints => generated 349 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:10:33.337417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f4856-a99f-44d6-ad7d-03ab856161b9 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:33.394825 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55364818-9fc8-415f-9c26-8ecf260da420 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:33.397483 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:33.463281 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7026a2-d91c-45df-a426-5744f83b2583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:33.463957 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7026a2-d91c-45df-a426-5744f83b2583 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:33.464308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7026a2-d91c-45df-a426-5744f83b2583 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:33.464591 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7026a2-d91c-45df-a426-5744f83b2583 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:33.465481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7026a2-d91c-45df-a426-5744f83b2583 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:33.467836 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7026a2-d91c-45df-a426-5744f83b2583 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:33.471563 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: WARNING keystone.server.flask.common [None req-5a7026a2-d91c-45df-a426-5744f83b2583 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 07 13:10:33.527791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.sql.core [None req-5a7026a2-d91c-45df-a426-5744f83b2583 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 13:10:33.527791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 07 13:10:33.527791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [parameters: {'user_id': '4720dc17c85a45aca2562b4214e70384', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 07 13:10:33.527791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=105315) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 13:10:33.542619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-5a7026a2-d91c-45df-a426-5744f83b2583 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 07 13:10:33.542619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:33.542619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 13:10:33.542619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 13:10:33.542619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 13:10:33.542619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 13:10:33.542619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 13:10:33.542619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 13:10:33.542619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 13:10:33.542619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application conn.query(q) Sep 07 13:10:33.542619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 13:10:33.542619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 13:10:33.542619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 13:10:33.542619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application result.read() Sep 07 13:10:33.542619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 13:10:33.544444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application next(self.gen) Sep 07 13:10:33.546927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 13:10:33.546927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application yield resource Sep 07 13:10:33.546927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 13:10:33.546927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application next(self.gen) Sep 07 13:10:33.546927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 13:10:33.546927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 13:10:33.546927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 13:10:33.546927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 13:10:33.546927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 13:10:33.546927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise self.value Sep 07 13:10:33.546927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 13:10:33.546927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 13:10:33.546927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 13:10:33.546927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application session.commit() Sep 07 13:10:33.546927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 13:10:33.546927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 13:10:33.546927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 13:10:33.550846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 13:10:33.550846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 13:10:33.550846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self.session.flush() Sep 07 13:10:33.550846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 13:10:33.550846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 13:10:33.550846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 13:10:33.550846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 13:10:33.550846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 13:10:33.550846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application compat.raise_( Sep 07 13:10:33.550846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 13:10:33.550846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception Sep 07 13:10:33.550846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 13:10:33.550846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 13:10:33.550846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 13:10:33.550846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 13:10:33.550846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 13:10:33.550846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 13:10:33.554521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 13:10:33.554521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 13:10:33.554521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 07 13:10:33.554521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 07 13:10:33.554521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 13:10:33.554521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 13:10:33.554521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 13:10:33.554521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 13:10:33.554521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 13:10:33.554521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 13:10:33.554521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 13:10:33.554521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 13:10:33.554521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 13:10:33.554521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 13:10:33.554521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 13:10:33.554521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception Sep 07 13:10:33.555811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 13:10:33.555811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 13:10:33.555811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 13:10:33.555811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 13:10:33.555811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 13:10:33.555811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 13:10:33.555811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 13:10:33.555811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application conn.query(q) Sep 07 13:10:33.555811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 13:10:33.555811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 13:10:33.555811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 13:10:33.555811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application result.read() Sep 07 13:10:33.555811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 13:10:33.555811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 13:10:33.555811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 13:10:33.555811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 13:10:33.556986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 13:10:33.556986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 13:10:33.556986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 13:10:33.556986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 13:10:33.556986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 13:10:33.556986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 07 13:10:33.556986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [parameters: {'user_id': '4720dc17c85a45aca2562b4214e70384', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 07 13:10:33.556986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 13:10:33.556986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:33.556986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:33.556986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:33.556986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:33.556986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:33.556986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:33.556986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:33.558139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:33.558139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:33.558139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:33.558139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:33.558139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:33.558139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:33.558139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:33.558139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 07 13:10:33.558139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 07 13:10:33.558139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:33.558139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:33.558139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:10:33.558139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:33.558139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:10:33.558139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:33.558139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 07 13:10:33.558139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 07 13:10:33.559349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 07 13:10:33.559349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 07 13:10:33.559349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 13:10:33.559349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 13:10:33.559349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 07 13:10:33.559349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:33.559349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 491/967] 10.222.0.62 () {60 vars in 1131 bytes} [Wed Sep 7 13:10:33 2022] POST /identity/v3/users => generated 160 bytes in 302 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:33.559349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7026a2-d91c-45df-a426-5744f83b2583 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:33.585435 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:33.591806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f4856-a99f-44d6-ad7d-03ab856161b9 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:33.641325 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc73ffd-d38b-4831-aa4b-ec4e22512b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:33.642252 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc73ffd-d38b-4831-aa4b-ec4e22512b94 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:33.642575 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc73ffd-d38b-4831-aa4b-ec4e22512b94 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:33.642849 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc73ffd-d38b-4831-aa4b-ec4e22512b94 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:33.643662 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc73ffd-d38b-4831-aa4b-ec4e22512b94 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:33.643966 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc73ffd-d38b-4831-aa4b-ec4e22512b94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:33.646041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc73ffd-d38b-4831-aa4b-ec4e22512b94 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:33.674565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 477/968] 10.222.0.62 () {56 vars in 1149 bytes} [Wed Sep 7 13:10:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:33.676550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc73ffd-d38b-4831-aa4b-ec4e22512b94 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:33.748670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7026a2-d91c-45df-a426-5744f83b2583 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:33.809115 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440fd47-0fcb-4496-92e6-fd38170c1320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:33.810227 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440fd47-0fcb-4496-92e6-fd38170c1320 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:33.810727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440fd47-0fcb-4496-92e6-fd38170c1320 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:33.811817 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440fd47-0fcb-4496-92e6-fd38170c1320 None admin] PATH_INFO: `/v3/services/compute` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:33.814007 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440fd47-0fcb-4496-92e6-fd38170c1320 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:33.817129 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440fd47-0fcb-4496-92e6-fd38170c1320 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:33.837466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-a440fd47-0fcb-4496-92e6-fd38170c1320 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:10:33.837466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:33.837466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:33.837466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:33.837466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:33.837466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:33.837466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:33.837466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:33.837466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:10:33.837466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:33.837466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:33.837466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:33.837466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:33.837466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:33.837466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:33.837466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:33.837466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:33.838691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:33.838691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:33.838691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:33.838691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:33.838691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:33.838691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:33.838691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:33.838691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:33.838691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:33.838691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:33.838691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:33.838691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:33.838691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:33.838691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:33.838691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:33.838691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:33.840133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:33.840133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:33.840133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:33.840133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:33.840133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:33.840133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:33.840133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:33.840133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:33.840133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:33.840133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:33.840133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:33.840133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:33.840133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:10:33.840133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:33.840133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 492/969] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:10:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:33.840979 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440fd47-0fcb-4496-92e6-fd38170c1320 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:33.893070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc73ffd-d38b-4831-aa4b-ec4e22512b94 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:33.899639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:33.967367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc4366-e529-4514-ab00-baa7772d16b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:33.968433 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc4366-e529-4514-ab00-baa7772d16b0 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:33.968860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc4366-e529-4514-ab00-baa7772d16b0 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:33.969270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc4366-e529-4514-ab00-baa7772d16b0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:33.982504 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc4366-e529-4514-ab00-baa7772d16b0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:33.986680 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc4366-e529-4514-ab00-baa7772d16b0 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:34.011834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-93bc4366-e529-4514-ab00-baa7772d16b0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 13:10:34.011834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:34.011834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:34.011834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:34.011834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:34.011834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:34.011834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:34.011834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:34.011834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:34.011834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:34.011834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:34.011834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:34.011834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:10:34.011834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:10:34.011834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:10:34.011834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:34.013443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:34.014873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:34.014873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:34.014873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:10:34.014873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:10:34.014873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:10:34.014873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:10:34.014873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:10:34.014873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:10:34.014873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 13:10:34.014873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:34.014873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 478/970] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:10:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:34.017346 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc4366-e529-4514-ab00-baa7772d16b0 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:34.035636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440fd47-0fcb-4496-92e6-fd38170c1320 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:34.059839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:34.104875 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c958ae-0eed-4714-a571-d81777b7aaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:34.106288 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c958ae-0eed-4714-a571-d81777b7aaab None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:34.106681 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c958ae-0eed-4714-a571-d81777b7aaab None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:34.107138 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c958ae-0eed-4714-a571-d81777b7aaab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:34.200939 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc4366-e529-4514-ab00-baa7772d16b0 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:34.232428 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c958ae-0eed-4714-a571-d81777b7aaab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:34.234267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c958ae-0eed-4714-a571-d81777b7aaab None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:34.273012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd6359-6f94-48d9-a33d-a495cfe290ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:34.273861 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd6359-6f94-48d9-a33d-a495cfe290ff None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:34.274297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd6359-6f94-48d9-a33d-a495cfe290ff None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:34.274879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd6359-6f94-48d9-a33d-a495cfe290ff None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:34.275825 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd6359-6f94-48d9-a33d-a495cfe290ff None admin] RBAC: Adding query filter params (name=compute) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:34.276342 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd6359-6f94-48d9-a33d-a495cfe290ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:34.277579 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:34.278133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd6359-6f94-48d9-a33d-a495cfe290ff None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:34.286269 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 479/971] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 7 13:10:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:34.287379 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd6359-6f94-48d9-a33d-a495cfe290ff None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:34.427152 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 493/972] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:10:33 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:10:34.428607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c958ae-0eed-4714-a571-d81777b7aaab None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:34.440020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:34.478545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd6359-6f94-48d9-a33d-a495cfe290ff None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:34.548975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c323019-318e-47a0-983b-3d7eb2386645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:34.549896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c323019-318e-47a0-983b-3d7eb2386645 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:34.550447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c323019-318e-47a0-983b-3d7eb2386645 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:34.550858 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c323019-318e-47a0-983b-3d7eb2386645 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:34.552299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c323019-318e-47a0-983b-3d7eb2386645 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:34.552795 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c323019-318e-47a0-983b-3d7eb2386645 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:34.554592 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c323019-318e-47a0-983b-3d7eb2386645 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:34.609416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 480/973] 10.222.0.62 () {56 vars in 1101 bytes} [Wed Sep 7 13:10:34 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:34.610883 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c323019-318e-47a0-983b-3d7eb2386645 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:34.656623 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c958ae-0eed-4714-a571-d81777b7aaab None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:34.704084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:34.723017 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22644651-068a-42d2-ae25-d6dcea616ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:34.724105 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22644651-068a-42d2-ae25-d6dcea616ffa None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:34.724601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22644651-068a-42d2-ae25-d6dcea616ffa None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:34.725098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22644651-068a-42d2-ae25-d6dcea616ffa None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:34.726199 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22644651-068a-42d2-ae25-d6dcea616ffa None admin] RBAC: Adding query filter params (type=compute) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:34.726737 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22644651-068a-42d2-ae25-d6dcea616ffa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:34.729251 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22644651-068a-42d2-ae25-d6dcea616ffa None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:34.747638 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 494/974] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 7 13:10:34 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:34.829984 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c323019-318e-47a0-983b-3d7eb2386645 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:34.899237 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7013d08-990f-42c3-870d-2a70e6c615e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:34.900239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7013d08-990f-42c3-870d-2a70e6c615e7 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:34.900673 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7013d08-990f-42c3-870d-2a70e6c615e7 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:34.901101 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7013d08-990f-42c3-870d-2a70e6c615e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:34.914323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:34.923247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7013d08-990f-42c3-870d-2a70e6c615e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:34.926007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7013d08-990f-42c3-870d-2a70e6c615e7 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:34.954330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-e7013d08-990f-42c3-870d-2a70e6c615e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:34.954330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:34.954330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:34.954330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:34.954330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:34.954330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:34.954330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:34.954330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:34.954330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:34.954330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:34.954330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:34.954330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:34.954330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:34.954330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:34.954330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:34.954330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:34.956804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:34.956804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:34.956804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:34.956804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:34.956804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:34.956804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:34.956804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:34.956804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:34.956804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:34.956804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:34.956804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:34.956804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:34.956804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:34.956804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:34.956804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:34.956804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:34.956804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:34.959356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:34.959356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:34.959356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:34.959356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 481/975] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:34.966352 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22644651-068a-42d2-ae25-d6dcea616ffa None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:35.175343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22644651-068a-42d2-ae25-d6dcea616ffa None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:35.241434 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036184e-71c1-4663-a5b8-e287ca160c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:35.242609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036184e-71c1-4663-a5b8-e287ca160c92 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:35.243131 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036184e-71c1-4663-a5b8-e287ca160c92 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:35.243621 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036184e-71c1-4663-a5b8-e287ca160c92 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:35.244979 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036184e-71c1-4663-a5b8-e287ca160c92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:35.245343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036184e-71c1-4663-a5b8-e287ca160c92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:35.247971 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036184e-71c1-4663-a5b8-e287ca160c92 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:35.279092 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 495/976] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:35.285106 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7013d08-990f-42c3-870d-2a70e6c615e7 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:35.485796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7013d08-990f-42c3-870d-2a70e6c615e7 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:35.533988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb55335-3d75-439e-8b19-8f1cd291099d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:35.535142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb55335-3d75-439e-8b19-8f1cd291099d None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:35.535686 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb55335-3d75-439e-8b19-8f1cd291099d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:35.536055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb55335-3d75-439e-8b19-8f1cd291099d None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:35.545193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb55335-3d75-439e-8b19-8f1cd291099d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:35.546887 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb55335-3d75-439e-8b19-8f1cd291099d None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:35.561636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-bbb55335-3d75-439e-8b19-8f1cd291099d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:35.561636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:35.561636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:35.561636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:35.561636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:35.561636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:35.561636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:35.561636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:35.561636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:35.561636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:35.561636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:35.561636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:35.561636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:10:35.561636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:10:35.561636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:10:35.572965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:10:35.572965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:35.572965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:35.572965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:35.572965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:35.572965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:35.572965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:35.572965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:35.572965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:35.572965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:35.572965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:35.572965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:35.572965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:35.572965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:35.572965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:35.572965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:35.572965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:35.575079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:10:35.575079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:10:35.575079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:10:35.575079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:10:35.575079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:10:35.575079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:10:35.575079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:35.575079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:35.575079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 482/977] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:10:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:35.580561 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036184e-71c1-4663-a5b8-e287ca160c92 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:35.678271 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:35.790369 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036184e-71c1-4663-a5b8-e287ca160c92 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:35.866747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a9f38-8ee9-4f5a-9541-34a9a7e3d74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:35.868028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a9f38-8ee9-4f5a-9541-34a9a7e3d74d None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:35.868523 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a9f38-8ee9-4f5a-9541-34a9a7e3d74d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:35.869008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a9f38-8ee9-4f5a-9541-34a9a7e3d74d None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:35.870596 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a9f38-8ee9-4f5a-9541-34a9a7e3d74d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:35.871143 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a9f38-8ee9-4f5a-9541-34a9a7e3d74d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:35.874550 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a9f38-8ee9-4f5a-9541-34a9a7e3d74d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:35.907022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 496/978] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:10:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:35.911693 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb55335-3d75-439e-8b19-8f1cd291099d None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:35.983961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b320d3-bd00-4840-bf0a-4788f37e08d3 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:35.984263 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b320d3-bd00-4840-bf0a-4788f37e08d3 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:35.984538 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b320d3-bd00-4840-bf0a-4788f37e08d3 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:35.986185 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 497/979] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:10:35.993250 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5d4c8-3534-44ca-a572-f8d60789b62b None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:35.993783 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5d4c8-3534-44ca-a572-f8d60789b62b None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:35.994322 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5d4c8-3534-44ca-a572-f8d60789b62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:36.020387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:36.023853 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-7fe5d4c8-3534-44ca-a572-f8d60789b62b None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:36.026454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7fe5d4c8-3534-44ca-a572-f8d60789b62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:36.120839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb55335-3d75-439e-8b19-8f1cd291099d None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:36.151071 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 498/980] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:10:36.189322 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc1ff6-6025-404a-aeda-8d6b085dcc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:36.190357 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc1ff6-6025-404a-aeda-8d6b085dcc9a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:36.190851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc1ff6-6025-404a-aeda-8d6b085dcc9a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:36.191378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc1ff6-6025-404a-aeda-8d6b085dcc9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:36.192775 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc1ff6-6025-404a-aeda-8d6b085dcc9a None admin] RBAC: Adding query filter params (role.id=e88e2f39fcb94166851f6db7b9f9dfcb, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=8121ba4cf2cf44d9a5a55975ea85928c) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:36.193405 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc1ff6-6025-404a-aeda-8d6b085dcc9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:36.196099 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc1ff6-6025-404a-aeda-8d6b085dcc9a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:36.209030 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 483/981] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:10:35 2022] GET /identity/v3/role_assignments?user.id=8121ba4cf2cf44d9a5a55975ea85928c&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=e88e2f39fcb94166851f6db7b9f9dfcb => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:36.429511 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5d4c8-3534-44ca-a572-f8d60789b62b None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:36.488042 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:36.614651 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5d4c8-3534-44ca-a572-f8d60789b62b None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:36.682099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b688835-f07c-4975-8318-0bcaf5310241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:36.682954 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b688835-f07c-4975-8318-0bcaf5310241 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:36.683493 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b688835-f07c-4975-8318-0bcaf5310241 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:36.683972 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b688835-f07c-4975-8318-0bcaf5310241 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:36.684905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b688835-f07c-4975-8318-0bcaf5310241 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:36.688371 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b688835-f07c-4975-8318-0bcaf5310241 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:36.698925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-3b688835-f07c-4975-8318-0bcaf5310241 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:10:36.698925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:36.698925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:36.698925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:36.698925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:36.698925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:36.698925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:36.698925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:36.698925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:10:36.698925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:36.698925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:36.698925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:36.698925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:36.698925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:36.698925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:36.698925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:36.698925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:36.700533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:36.700533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:36.700533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:36.700533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:36.700533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:36.700533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:36.700533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:36.700533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:36.700533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:36.700533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:36.700533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:36.700533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:36.700533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:36.700533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:36.700533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:36.700533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:36.702810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:36.702810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:36.702810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:36.702810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:36.702810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:36.702810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:36.702810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:36.702810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:36.702810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:36.702810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:36.702810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:36.702810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:36.702810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:10:36.702810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:36.702810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 499/982] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 7 13:10:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:36.710210 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc1ff6-6025-404a-aeda-8d6b085dcc9a None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:36.907234 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc1ff6-6025-404a-aeda-8d6b085dcc9a None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:36.938404 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:36.979708 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049af267-c5bc-411c-b72e-76d0f092c1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:36.980736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049af267-c5bc-411c-b72e-76d0f092c1fb None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:36.981222 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049af267-c5bc-411c-b72e-76d0f092c1fb None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:36.981787 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049af267-c5bc-411c-b72e-76d0f092c1fb None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:36.983975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049af267-c5bc-411c-b72e-76d0f092c1fb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:36.984648 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049af267-c5bc-411c-b72e-76d0f092c1fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:36.987419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049af267-c5bc-411c-b72e-76d0f092c1fb None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:36.996926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 484/983] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 7 13:10:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:37.006540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b688835-f07c-4975-8318-0bcaf5310241 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:37.168322 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b688835-f07c-4975-8318-0bcaf5310241 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:37.171273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:37.226864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d70a4-3f31-4917-b476-8ae4ccb8a0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:37.227548 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d70a4-3f31-4917-b476-8ae4ccb8a0e8 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:37.227838 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d70a4-3f31-4917-b476-8ae4ccb8a0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:37.228160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d70a4-3f31-4917-b476-8ae4ccb8a0e8 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:37.228906 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d70a4-3f31-4917-b476-8ae4ccb8a0e8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:37.229207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d70a4-3f31-4917-b476-8ae4ccb8a0e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:37.230673 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d70a4-3f31-4917-b476-8ae4ccb8a0e8 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:37.242457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 500/984] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 7 13:10:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:37.247818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049af267-c5bc-411c-b72e-76d0f092c1fb None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:37.454077 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049af267-c5bc-411c-b72e-76d0f092c1fb None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:37.521445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c619915-66a3-46e2-9f5e-11ba171e8493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:37.522346 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c619915-66a3-46e2-9f5e-11ba171e8493 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:37.522771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c619915-66a3-46e2-9f5e-11ba171e8493 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:37.523309 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c619915-66a3-46e2-9f5e-11ba171e8493 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:37.524672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c619915-66a3-46e2-9f5e-11ba171e8493 None admin] RBAC: Adding query filter params (interface=admin, service_id=f09022ea4c56493fbc8c07ba240f0243, region_id=RegionOne) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:37.528333 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c619915-66a3-46e2-9f5e-11ba171e8493 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:37.531026 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c619915-66a3-46e2-9f5e-11ba171e8493 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:37.532459 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c509a-a7de-490f-a311-fc3fa7b98037 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:37.532766 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c509a-a7de-490f-a311-fc3fa7b98037 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:37.533074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c509a-a7de-490f-a311-fc3fa7b98037 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:37.535174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 501/985] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:10:37.540566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dca02-7a8e-49dd-9b74-5ecedab08c64 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:37.540913 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dca02-7a8e-49dd-9b74-5ecedab08c64 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:37.541113 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dca02-7a8e-49dd-9b74-5ecedab08c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:37.543430 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 485/986] 10.222.0.62 () {56 vars in 1249 bytes} [Wed Sep 7 13:10:37 2022] GET /identity/v3/endpoints?service_id=f09022ea4c56493fbc8c07ba240f0243&interface=admin®ion_id=RegionOne => generated 193 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:37.551820 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c619915-66a3-46e2-9f5e-11ba171e8493 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:37.567826 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-cb5dca02-7a8e-49dd-9b74-5ecedab08c64 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:37.570047 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-cb5dca02-7a8e-49dd-9b74-5ecedab08c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:37.617957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:37.667287 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 502/987] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:10:37.705124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c619915-66a3-46e2-9f5e-11ba171e8493 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:37.745842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea04413-0d0e-4b50-974c-9d3d19ac3829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:37.746481 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea04413-0d0e-4b50-974c-9d3d19ac3829 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:37.746718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea04413-0d0e-4b50-974c-9d3d19ac3829 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:37.746892 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea04413-0d0e-4b50-974c-9d3d19ac3829 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:37.747533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea04413-0d0e-4b50-974c-9d3d19ac3829 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:37.749534 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea04413-0d0e-4b50-974c-9d3d19ac3829 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:37.763160 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 486/988] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 7 13:10:37 2022] GET /identity/v3/services => generated 1497 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:10:37.955939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5dca02-7a8e-49dd-9b74-5ecedab08c64 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:38.116230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:38.129002 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5dca02-7a8e-49dd-9b74-5ecedab08c64 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:38.188619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16deb3-3338-4234-9e1f-7d82e5245169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:38.189421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16deb3-3338-4234-9e1f-7d82e5245169 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:38.189853 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16deb3-3338-4234-9e1f-7d82e5245169 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:38.190361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16deb3-3338-4234-9e1f-7d82e5245169 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:38.191260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16deb3-3338-4234-9e1f-7d82e5245169 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:38.192908 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16deb3-3338-4234-9e1f-7d82e5245169 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:38.269137 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 503/989] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 7 13:10:37 2022] POST /identity/v3/services => generated 242 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:10:39.211987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383fa45-5d5b-48b7-bfd5-78441bba39d6 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:39.212800 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383fa45-5d5b-48b7-bfd5-78441bba39d6 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:39.212800 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383fa45-5d5b-48b7-bfd5-78441bba39d6 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:39.215030 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 487/990] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:39.223815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb6584-8fa5-4e4d-a380-b1bdf789869f None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:39.224376 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb6584-8fa5-4e4d-a380-b1bdf789869f None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:39.224712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb6584-8fa5-4e4d-a380-b1bdf789869f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:39.256657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-30eb6584-8fa5-4e4d-a380-b1bdf789869f None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:39.259265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-30eb6584-8fa5-4e4d-a380-b1bdf789869f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:39.371369 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 504/991] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:39.676722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383fa45-5d5b-48b7-bfd5-78441bba39d6 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:39.864212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383fa45-5d5b-48b7-bfd5-78441bba39d6 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:39.874704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:39.907236 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725171e-cc68-4c16-aaf4-9a806a37e65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:39.908029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725171e-cc68-4c16-aaf4-9a806a37e65a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:39.908316 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725171e-cc68-4c16-aaf4-9a806a37e65a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:39.908757 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725171e-cc68-4c16-aaf4-9a806a37e65a None admin] PATH_INFO: `/v3/services/image` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:39.909655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1725171e-cc68-4c16-aaf4-9a806a37e65a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:39.911418 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1725171e-cc68-4c16-aaf4-9a806a37e65a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:39.922258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-1725171e-cc68-4c16-aaf4-9a806a37e65a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:10:39.922258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:39.922258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:39.922258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:39.922258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:39.922258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:39.922258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:39.922258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:39.922258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:10:39.922258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:39.922258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:39.922258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:39.922258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:39.922258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:39.922258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:39.922258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:39.922258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:39.923455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:39.923455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:39.923455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:39.923455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:39.923455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:39.923455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:39.923455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:39.923455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:39.923455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:39.923455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:39.923455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:39.923455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:39.923455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:39.923455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:39.923455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:39.923455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:39.928241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:39.928241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:39.928241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:39.928241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:39.928241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:39.928241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:39.928241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:39.928241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:39.928241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:39.928241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:39.928241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:39.928241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:39.928241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:10:39.928241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:39.928241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 488/992] 10.222.0.62 () {56 vars in 1105 bytes} [Wed Sep 7 13:10:39 2022] GET /identity/v3/services/image => generated 86 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:39.930277 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb6584-8fa5-4e4d-a380-b1bdf789869f None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:40.030825 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:40.054300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb6584-8fa5-4e4d-a380-b1bdf789869f None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:40.098901 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86985e4b-f217-4df2-b026-6706ddb8d9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:40.099563 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86985e4b-f217-4df2-b026-6706ddb8d9d0 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:40.099930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86985e4b-f217-4df2-b026-6706ddb8d9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:40.100216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86985e4b-f217-4df2-b026-6706ddb8d9d0 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:40.100845 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86985e4b-f217-4df2-b026-6706ddb8d9d0 None admin] RBAC: Adding query filter params (name=image) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:40.101151 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86985e4b-f217-4df2-b026-6706ddb8d9d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:40.102622 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86985e4b-f217-4df2-b026-6706ddb8d9d0 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:40.117167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725171e-cc68-4c16-aaf4-9a806a37e65a None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:40.118222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 505/993] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:40.258928 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a92aab-fd4d-40e5-ae82-d8381bba5191 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:40.259778 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a92aab-fd4d-40e5-ae82-d8381bba5191 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:40.260030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a92aab-fd4d-40e5-ae82-d8381bba5191 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:40.263026 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 506/994] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:40.269466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d185c-d616-42fa-abbd-7653bd38d911 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:40.269757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d185c-d616-42fa-abbd-7653bd38d911 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:40.270093 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d185c-d616-42fa-abbd-7653bd38d911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:40.273318 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725171e-cc68-4c16-aaf4-9a806a37e65a None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:40.290516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-604d185c-d616-42fa-abbd-7653bd38d911 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:40.292007 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-604d185c-d616-42fa-abbd-7653bd38d911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:40.333350 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd250d1-45c3-433d-8493-62ab2609c2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:40.334272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd250d1-45c3-433d-8493-62ab2609c2e4 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:40.334717 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd250d1-45c3-433d-8493-62ab2609c2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:40.335271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd250d1-45c3-433d-8493-62ab2609c2e4 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:40.336506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd250d1-45c3-433d-8493-62ab2609c2e4 None admin] RBAC: Adding query filter params (type=image) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:40.337211 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd250d1-45c3-433d-8493-62ab2609c2e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:40.339309 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd250d1-45c3-433d-8493-62ab2609c2e4 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:40.359030 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 489/995] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:40 2022] GET /identity/v3/services?type=image => generated 122 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:40.399136 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 507/996] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:40 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:40.701840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd250d1-45c3-433d-8493-62ab2609c2e4 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:40.751653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:40.845271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd250d1-45c3-433d-8493-62ab2609c2e4 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:40.894186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a91a4b-0218-4076-b6e6-6b8ea541e431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:40.894967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a91a4b-0218-4076-b6e6-6b8ea541e431 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:40.895372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a91a4b-0218-4076-b6e6-6b8ea541e431 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:40.895656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a91a4b-0218-4076-b6e6-6b8ea541e431 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:40.896347 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a91a4b-0218-4076-b6e6-6b8ea541e431 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:40.897909 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a91a4b-0218-4076-b6e6-6b8ea541e431 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:40.911156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-09a91a4b-0218-4076-b6e6-6b8ea541e431 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:10:40.911156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:40.911156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:40.911156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:40.911156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:40.911156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:40.911156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:40.911156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:40.911156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:10:40.911156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:40.911156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:40.911156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:40.911156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:40.911156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:40.911156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:40.911156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:40.911156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:40.912595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:40.912595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:40.912595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:40.912595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:40.912595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:40.912595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:40.912595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:40.912595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:40.912595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:40.912595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:40.912595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:40.912595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:40.912595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:40.912595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:40.912595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:40.912595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:40.914058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:40.914058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:40.914058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:40.914058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:40.914058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:40.914058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:40.914058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:40.914058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:40.914058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:40.914058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:40.914058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:40.914058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:40.914058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 13:10:40.914058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:40.922011 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 490/997] 10.222.0.62 () {56 vars in 1126 bytes} [Wed Sep 7 13:10:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:40.922208 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d185c-d616-42fa-abbd-7653bd38d911 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:41.016959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:41.051428 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d185c-d616-42fa-abbd-7653bd38d911 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:41.102923 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfab83-881e-4cae-afbe-c7551ff40ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:41.103754 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfab83-881e-4cae-afbe-c7551ff40ca2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:41.104031 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfab83-881e-4cae-afbe-c7551ff40ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:41.104310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfab83-881e-4cae-afbe-c7551ff40ca2 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:41.105000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfab83-881e-4cae-afbe-c7551ff40ca2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:41.105305 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfab83-881e-4cae-afbe-c7551ff40ca2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:41.106809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfab83-881e-4cae-afbe-c7551ff40ca2 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:41.115335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 508/998] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 7 13:10:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:41.121144 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a91a4b-0218-4076-b6e6-6b8ea541e431 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:41.315810 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:41.323780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a91a4b-0218-4076-b6e6-6b8ea541e431 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:41.386097 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e831a8c-a324-43e4-b831-dfde524a4249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:41.386714 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e831a8c-a324-43e4-b831-dfde524a4249 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:41.387014 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e831a8c-a324-43e4-b831-dfde524a4249 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:41.387295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e831a8c-a324-43e4-b831-dfde524a4249 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:41.387941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e831a8c-a324-43e4-b831-dfde524a4249 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:41.388261 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e831a8c-a324-43e4-b831-dfde524a4249 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:41.390591 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e831a8c-a324-43e4-b831-dfde524a4249 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:41.402219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 491/999] 10.222.0.62 () {56 vars in 1122 bytes} [Wed Sep 7 13:10:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:41.403560 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfab83-881e-4cae-afbe-c7551ff40ca2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:41.494603 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0796ac-b102-496a-bd88-42272113c32a None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:41.495196 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0796ac-b102-496a-bd88-42272113c32a None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:41.495531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0796ac-b102-496a-bd88-42272113c32a None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:41.497637 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 492/1000] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:10:41.504964 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c1484-16b2-4631-b431-6957997a6a8c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:41.505391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c1484-16b2-4631-b431-6957997a6a8c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:41.505673 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c1484-16b2-4631-b431-6957997a6a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:41.531161 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-126c1484-16b2-4631-b431-6957997a6a8c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:41.533291 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-126c1484-16b2-4631-b431-6957997a6a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:41.558959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfab83-881e-4cae-afbe-c7551ff40ca2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:41.611372 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57480869-d0a7-4e2a-aca7-b6ccdc20cac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:41.611909 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57480869-d0a7-4e2a-aca7-b6ccdc20cac8 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:41.612768 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57480869-d0a7-4e2a-aca7-b6ccdc20cac8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:41.613046 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57480869-d0a7-4e2a-aca7-b6ccdc20cac8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:41.613735 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57480869-d0a7-4e2a-aca7-b6ccdc20cac8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:41.615285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57480869-d0a7-4e2a-aca7-b6ccdc20cac8 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:41.628385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 493/1001] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:10:41.707686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:41.717837 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 509/1002] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:10:41 2022] POST /identity/v3/endpoints => generated 325 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:10:41.933447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c1484-16b2-4631-b431-6957997a6a8c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:42.103270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:42.122220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c1484-16b2-4631-b431-6957997a6a8c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:42.205412 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69c081-af61-4f61-8d8e-1621d0100079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:42.206416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69c081-af61-4f61-8d8e-1621d0100079 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:42.206906 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69c081-af61-4f61-8d8e-1621d0100079 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:42.207426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69c081-af61-4f61-8d8e-1621d0100079 None admin] PATH_INFO: `/v3/services/compute` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:42.208745 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69c081-af61-4f61-8d8e-1621d0100079 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:42.211390 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69c081-af61-4f61-8d8e-1621d0100079 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:42.229731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-bb69c081-af61-4f61-8d8e-1621d0100079 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:10:42.229731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:42.229731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:42.229731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:42.229731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:42.229731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:42.229731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:42.229731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:42.229731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:10:42.229731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:42.229731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:42.229731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:42.229731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:42.229731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:42.229731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:42.229731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:42.229731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:42.230886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:42.230886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:42.230886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:42.230886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:42.230886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:42.230886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:42.230886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:42.230886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:42.230886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:42.230886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:42.230886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:42.230886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:42.230886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:42.230886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:42.230886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:42.230886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:42.231873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:42.231873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:42.231873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:42.231873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:42.231873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:42.231873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:42.231873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:42.231873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:42.231873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:42.231873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:42.231873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:42.231873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:42.231873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:10:42.231873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:42.236745 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 494/1003] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:10:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:42.239733 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57480869-d0a7-4e2a-aca7-b6ccdc20cac8 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:42.398687 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57480869-d0a7-4e2a-aca7-b6ccdc20cac8 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:42.447630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf532ae-0a2c-488d-a13e-5ea6923584c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:42.448357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf532ae-0a2c-488d-a13e-5ea6923584c2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:42.448693 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf532ae-0a2c-488d-a13e-5ea6923584c2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:42.449077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf532ae-0a2c-488d-a13e-5ea6923584c2 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:42.449828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf532ae-0a2c-488d-a13e-5ea6923584c2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:42.450175 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf532ae-0a2c-488d-a13e-5ea6923584c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:42.452528 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf532ae-0a2c-488d-a13e-5ea6923584c2 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:42.459864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 510/1004] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 7 13:10:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:42.467845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69c081-af61-4f61-8d8e-1621d0100079 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:42.699607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69c081-af61-4f61-8d8e-1621d0100079 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:42.775802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6865c88-abd9-421e-b2bb-dc1b7d2ddb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:42.776813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6865c88-abd9-421e-b2bb-dc1b7d2ddb43 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:42.777283 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6865c88-abd9-421e-b2bb-dc1b7d2ddb43 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:42.777752 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6865c88-abd9-421e-b2bb-dc1b7d2ddb43 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:42.778925 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6865c88-abd9-421e-b2bb-dc1b7d2ddb43 None admin] RBAC: Adding query filter params (type=compute) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:42.780047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6865c88-abd9-421e-b2bb-dc1b7d2ddb43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:42.783638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6865c88-abd9-421e-b2bb-dc1b7d2ddb43 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:42.799382 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 495/1005] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 7 13:10:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:42.805113 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf532ae-0a2c-488d-a13e-5ea6923584c2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:42.940294 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:43.020094 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf532ae-0a2c-488d-a13e-5ea6923584c2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:43.070037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542dcc22-28c6-4011-8807-e4cbb00b44b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:43.070304 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542dcc22-28c6-4011-8807-e4cbb00b44b5 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:43.070645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542dcc22-28c6-4011-8807-e4cbb00b44b5 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:43.070819 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542dcc22-28c6-4011-8807-e4cbb00b44b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:43.071571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542dcc22-28c6-4011-8807-e4cbb00b44b5 None admin] RBAC: Adding query filter params (interface=public, service_id=c6a36a322d6f41d1a46497d537d43dee, region_id=RegionOne) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:43.071875 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542dcc22-28c6-4011-8807-e4cbb00b44b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:43.073348 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542dcc22-28c6-4011-8807-e4cbb00b44b5 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:43.087038 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 511/1006] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Sep 7 13:10:42 2022] GET /identity/v3/endpoints?service_id=c6a36a322d6f41d1a46497d537d43dee&interface=public®ion_id=RegionOne => generated 194 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:43.093002 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6865c88-abd9-421e-b2bb-dc1b7d2ddb43 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:43.128799 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:43.132760 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bbd81-3d74-4bcb-8d73-5fd2a47f84e6 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:43.133130 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bbd81-3d74-4bcb-8d73-5fd2a47f84e6 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:43.133450 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bbd81-3d74-4bcb-8d73-5fd2a47f84e6 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:43.135255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 512/1007] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:43.141744 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b406eed-abf4-4431-8451-0867438ebe73 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:43.142120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b406eed-abf4-4431-8451-0867438ebe73 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:43.144067 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b406eed-abf4-4431-8451-0867438ebe73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:43.183511 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-2b406eed-abf4-4431-8451-0867438ebe73 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:43.185498 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-2b406eed-abf4-4431-8451-0867438ebe73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:43.291927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 513/1008] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:43 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:43.319970 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6865c88-abd9-421e-b2bb-dc1b7d2ddb43 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:43.395790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc3762-8b7c-4d1c-8ef7-45ad6563e068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:43.396830 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc3762-8b7c-4d1c-8ef7-45ad6563e068 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:43.397224 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc3762-8b7c-4d1c-8ef7-45ad6563e068 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:43.397714 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc3762-8b7c-4d1c-8ef7-45ad6563e068 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:43.398861 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafc3762-8b7c-4d1c-8ef7-45ad6563e068 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:43.401833 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafc3762-8b7c-4d1c-8ef7-45ad6563e068 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:43.416141 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 496/1009] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 7 13:10:43 2022] GET /identity/v3/services => generated 1727 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:10:43.640759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b406eed-abf4-4431-8451-0867438ebe73 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:43.765508 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:43.857153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b406eed-abf4-4431-8451-0867438ebe73 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:43.928433 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34af66-d080-4dec-9ff5-d4b89e708e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:43.929508 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34af66-d080-4dec-9ff5-d4b89e708e1b None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:43.929996 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34af66-d080-4dec-9ff5-d4b89e708e1b None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:43.930503 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34af66-d080-4dec-9ff5-d4b89e708e1b None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:43.931792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34af66-d080-4dec-9ff5-d4b89e708e1b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:43.934557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34af66-d080-4dec-9ff5-d4b89e708e1b None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:44.077522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 514/1010] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 7 13:10:43 2022] POST /identity/v3/services => generated 242 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:10:44.550228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4014d82-0f5c-4454-b15e-02078aa31485 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:44.551105 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4014d82-0f5c-4454-b15e-02078aa31485 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:44.551246 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4014d82-0f5c-4454-b15e-02078aa31485 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:44.553205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 497/1011] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:44.559414 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5229eb3-12d4-43bf-a5cb-d86bca4b50e1 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:44.559873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5229eb3-12d4-43bf-a5cb-d86bca4b50e1 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:44.560272 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5229eb3-12d4-43bf-a5cb-d86bca4b50e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:44.583650 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-e5229eb3-12d4-43bf-a5cb-d86bca4b50e1 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:44.585113 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e5229eb3-12d4-43bf-a5cb-d86bca4b50e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:44.683386 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 515/1012] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:44 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:44.978384 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4014d82-0f5c-4454-b15e-02078aa31485 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:45.083972 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:45.127284 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4014d82-0f5c-4454-b15e-02078aa31485 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:45.198408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf91a9-4699-42af-93bc-ea2f2103cd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:45.199889 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf91a9-4699-42af-93bc-ea2f2103cd46 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:45.200642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf91a9-4699-42af-93bc-ea2f2103cd46 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:45.202777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf91a9-4699-42af-93bc-ea2f2103cd46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:45.204911 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf91a9-4699-42af-93bc-ea2f2103cd46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:45.206478 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf91a9-4699-42af-93bc-ea2f2103cd46 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:45.208577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 498/1013] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:45.214857 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5229eb3-12d4-43bf-a5cb-d86bca4b50e1 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:45.288688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:45.360259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5229eb3-12d4-43bf-a5cb-d86bca4b50e1 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:45.407014 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05167a-31e4-4e6c-8c8d-9e7c61d40e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:45.407664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05167a-31e4-4e6c-8c8d-9e7c61d40e96 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:45.407942 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05167a-31e4-4e6c-8c8d-9e7c61d40e96 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:45.408238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05167a-31e4-4e6c-8c8d-9e7c61d40e96 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:45.408935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05167a-31e4-4e6c-8c8d-9e7c61d40e96 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:45.410403 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05167a-31e4-4e6c-8c8d-9e7c61d40e96 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:45.543181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 516/1014] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 7 13:10:45 2022] POST /identity/v3/projects => generated 311 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:10:45.907309 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179d6ec-5279-487d-b22d-d2f6d9b168f8 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:45.907876 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179d6ec-5279-487d-b22d-d2f6d9b168f8 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:45.907876 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179d6ec-5279-487d-b22d-d2f6d9b168f8 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:45.909148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 499/1015] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:10:45.919193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12692aa8-58d5-478a-a1b2-99223031f7fe None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:45.919747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12692aa8-58d5-478a-a1b2-99223031f7fe None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:45.919747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12692aa8-58d5-478a-a1b2-99223031f7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:45.952163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-12692aa8-58d5-478a-a1b2-99223031f7fe None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:45.953543 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-12692aa8-58d5-478a-a1b2-99223031f7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:46.075902 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 517/1016] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:45 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:46.424931 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b658094-f3a1-4bd0-a2b7-052b1e8ce875 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:46.425551 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b658094-f3a1-4bd0-a2b7-052b1e8ce875 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:46.425830 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b658094-f3a1-4bd0-a2b7-052b1e8ce875 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:46.427727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 500/1017] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:46.433990 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85eee46-341e-497f-bd8f-05f316f9d8c6 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:46.434434 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85eee46-341e-497f-bd8f-05f316f9d8c6 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:46.434859 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85eee46-341e-497f-bd8f-05f316f9d8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:46.454026 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b658094-f3a1-4bd0-a2b7-052b1e8ce875 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:46.459274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-c85eee46-341e-497f-bd8f-05f316f9d8c6 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:46.461086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c85eee46-341e-497f-bd8f-05f316f9d8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:46.493015 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:46.575045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 518/1018] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:46 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:10:46.610153 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b658094-f3a1-4bd0-a2b7-052b1e8ce875 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:46.651954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36af8f7-71f9-457c-9718-3a7a17637c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:46.652628 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36af8f7-71f9-457c-9718-3a7a17637c3f None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:46.652920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36af8f7-71f9-457c-9718-3a7a17637c3f None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:46.653454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36af8f7-71f9-457c-9718-3a7a17637c3f None admin] PATH_INFO: `/v3/services/compute` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:46.653751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36af8f7-71f9-457c-9718-3a7a17637c3f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:46.655982 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36af8f7-71f9-457c-9718-3a7a17637c3f None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:46.667859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-e36af8f7-71f9-457c-9718-3a7a17637c3f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:10:46.667859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:46.667859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:46.667859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:46.667859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:46.667859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:46.667859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:46.667859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:46.667859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:10:46.667859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:46.667859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:46.667859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:46.667859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:46.667859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:46.667859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:46.667859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:46.667859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:46.669771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:46.669771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:46.669771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:46.669771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:46.669771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:46.669771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:46.669771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:46.669771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:46.669771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:46.669771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:46.669771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:46.669771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:46.669771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:46.669771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:46.669771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:46.669771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:46.670902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:46.670902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:46.670902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:46.670902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:46.670902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:46.670902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:46.670902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:46.670902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:46.670902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:46.670902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:46.670902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:46.670902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:46.670902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 13:10:46.670902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:46.670902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 501/1019] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:10:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:46.676985 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85eee46-341e-497f-bd8f-05f316f9d8c6 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:46.830804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85eee46-341e-497f-bd8f-05f316f9d8c6 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:46.884276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36af8f7-71f9-457c-9718-3a7a17637c3f None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:46.885460 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a5883-f2f4-449d-9cbb-a3d9a56187b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:46.886125 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a5883-f2f4-449d-9cbb-a3d9a56187b1 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:46.886411 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a5883-f2f4-449d-9cbb-a3d9a56187b1 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:46.886683 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a5883-f2f4-449d-9cbb-a3d9a56187b1 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:46.887389 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a5883-f2f4-449d-9cbb-a3d9a56187b1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:46.887695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a5883-f2f4-449d-9cbb-a3d9a56187b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:46.889817 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a5883-f2f4-449d-9cbb-a3d9a56187b1 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:46.901182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 519/1020] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 7 13:10:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:46.911165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a5883-f2f4-449d-9cbb-a3d9a56187b1 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:46.913574 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:46.948692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:47.052120 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36af8f7-71f9-457c-9718-3a7a17637c3f None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:47.052378 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a5883-f2f4-449d-9cbb-a3d9a56187b1 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:47.097889 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a724212-e413-4399-871d-b8e2c8543c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:47.098951 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a724212-e413-4399-871d-b8e2c8543c53 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:47.099494 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a724212-e413-4399-871d-b8e2c8543c53 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:47.099985 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a724212-e413-4399-871d-b8e2c8543c53 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:47.101208 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a724212-e413-4399-871d-b8e2c8543c53 None admin] RBAC: Adding query filter params (type=compute) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:47.101764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a724212-e413-4399-871d-b8e2c8543c53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:47.104405 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a724212-e413-4399-871d-b8e2c8543c53 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:47.110011 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef6ee5-7ed1-4840-9157-0cfb14695a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:47.111085 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef6ee5-7ed1-4840-9157-0cfb14695a9f None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:47.111593 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef6ee5-7ed1-4840-9157-0cfb14695a9f None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:47.112069 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef6ee5-7ed1-4840-9157-0cfb14695a9f None admin] PATH_INFO: `/v3/services/image` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:47.112884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef6ee5-7ed1-4840-9157-0cfb14695a9f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:47.113361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 520/1021] 10.222.0.62 () {56 vars in 1112 bytes} [Wed Sep 7 13:10:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:47.114997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef6ee5-7ed1-4840-9157-0cfb14695a9f None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:47.122277 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a724212-e413-4399-871d-b8e2c8543c53 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:47.128630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-b5ef6ee5-7ed1-4840-9157-0cfb14695a9f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:10:47.128630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:47.128630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:47.128630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:47.128630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:47.128630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:47.128630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:47.128630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:47.128630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:10:47.128630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:47.128630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:47.128630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:47.128630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:47.128630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:47.128630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:47.128630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:47.128630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:47.130586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:47.130586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:47.130586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:47.130586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:47.130586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:47.130586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:47.130586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:47.130586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:47.130586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:47.130586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:47.130586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:47.130586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:47.130586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:47.130586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:47.130586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:47.130586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:47.132313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:47.132313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:47.132313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:47.132313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:47.132313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:47.132313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:47.132313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:47.132313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:47.132313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:47.132313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:47.132313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:47.132313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:47.132313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:10:47.132313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:47.132313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 502/1022] 10.222.0.62 () {56 vars in 1105 bytes} [Wed Sep 7 13:10:46 2022] GET /identity/v3/services/image => generated 86 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:47.137656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef6ee5-7ed1-4840-9157-0cfb14695a9f None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:47.269200 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:47.271806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:47.299724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a724212-e413-4399-871d-b8e2c8543c53 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:47.303366 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef6ee5-7ed1-4840-9157-0cfb14695a9f None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:47.351270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e81eaf-b8d3-4429-8bc6-aac202aac6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:47.352182 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e81eaf-b8d3-4429-8bc6-aac202aac6ce None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:47.352462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e81eaf-b8d3-4429-8bc6-aac202aac6ce None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:47.352847 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e81eaf-b8d3-4429-8bc6-aac202aac6ce None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:47.353551 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e81eaf-b8d3-4429-8bc6-aac202aac6ce None admin] RBAC: Adding query filter params (name=image) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:47.354543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e81eaf-b8d3-4429-8bc6-aac202aac6ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:47.356546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e81eaf-b8d3-4429-8bc6-aac202aac6ce None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:47.365010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 503/1023] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:47.370155 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3f324-6f39-450f-bed8-a44fa5a0e0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:47.371264 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3f324-6f39-450f-bed8-a44fa5a0e0e3 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:47.371752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3f324-6f39-450f-bed8-a44fa5a0e0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:47.371904 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e81eaf-b8d3-4429-8bc6-aac202aac6ce None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:47.372366 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3f324-6f39-450f-bed8-a44fa5a0e0e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:47.373702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3f324-6f39-450f-bed8-a44fa5a0e0e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:47.376582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3f324-6f39-450f-bed8-a44fa5a0e0e3 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:47.472124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 521/1024] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:10:47 2022] POST /identity/v3/endpoints => generated 334 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:10:47.541111 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e81eaf-b8d3-4429-8bc6-aac202aac6ce None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:47.614639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf834853-dc82-42c2-827a-a2c31b1bcd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:47.615796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf834853-dc82-42c2-827a-a2c31b1bcd68 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:47.616252 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf834853-dc82-42c2-827a-a2c31b1bcd68 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:47.616816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf834853-dc82-42c2-827a-a2c31b1bcd68 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:47.617972 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf834853-dc82-42c2-827a-a2c31b1bcd68 None admin] RBAC: Adding query filter params (type=image) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:47.618512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf834853-dc82-42c2-827a-a2c31b1bcd68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:47.621484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf834853-dc82-42c2-827a-a2c31b1bcd68 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:47.626185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:47.631726 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 504/1025] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:47.642643 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3f324-6f39-450f-bed8-a44fa5a0e0e3 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:47.838859 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3f324-6f39-450f-bed8-a44fa5a0e0e3 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:47.892616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3973cdd-c2ae-48d9-ad03-90f533529b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:47.893386 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3973cdd-c2ae-48d9-ad03-90f533529b6b None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:47.893677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3973cdd-c2ae-48d9-ad03-90f533529b6b None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:47.894070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3973cdd-c2ae-48d9-ad03-90f533529b6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:47.894815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3973cdd-c2ae-48d9-ad03-90f533529b6b None admin] RBAC: Adding query filter params (interface=public, service_id=da84e222989c4d59b0a85d9ffafd0395, region_id=RegionOne) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:47.895160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3973cdd-c2ae-48d9-ad03-90f533529b6b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:47.896557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3973cdd-c2ae-48d9-ad03-90f533529b6b None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:47.903108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 522/1026] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Sep 7 13:10:47 2022] GET /identity/v3/endpoints?service_id=da84e222989c4d59b0a85d9ffafd0395&interface=public®ion_id=RegionOne => generated 194 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:47.910104 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf834853-dc82-42c2-827a-a2c31b1bcd68 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:48.036983 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf834853-dc82-42c2-827a-a2c31b1bcd68 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:48.076696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea0628-fc15-47a2-b926-c2bb2d53413a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:48.077687 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea0628-fc15-47a2-b926-c2bb2d53413a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:48.078116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea0628-fc15-47a2-b926-c2bb2d53413a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:48.078401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea0628-fc15-47a2-b926-c2bb2d53413a None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:48.079061 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ea0628-fc15-47a2-b926-c2bb2d53413a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:48.080721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ea0628-fc15-47a2-b926-c2bb2d53413a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:48.088191 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 505/1027] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 7 13:10:47 2022] GET /identity/v3/services => generated 1957 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:10:48.708982 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6640f4a-8c4b-40e4-a069-f270e820df1d None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:48.709757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6640f4a-8c4b-40e4-a069-f270e820df1d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:48.709757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6640f4a-8c4b-40e4-a069-f270e820df1d None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:48.711500 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 523/1028] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:10:48.717761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413b32c-674e-4685-b0c4-859321ff17be None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:48.718162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413b32c-674e-4685-b0c4-859321ff17be None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:48.718315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413b32c-674e-4685-b0c4-859321ff17be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:48.739706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-6413b32c-674e-4685-b0c4-859321ff17be None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:48.740960 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6413b32c-674e-4685-b0c4-859321ff17be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:48.857020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:48.879572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 506/1029] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:10:49.168112 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6640f4a-8c4b-40e4-a069-f270e820df1d None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:49.175246 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:49.375896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6640f4a-8c4b-40e4-a069-f270e820df1d None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:49.440830 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7303b9-d281-4050-9a86-8a4de8477fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:49.441676 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7303b9-d281-4050-9a86-8a4de8477fd0 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:49.442079 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7303b9-d281-4050-9a86-8a4de8477fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:49.442726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7303b9-d281-4050-9a86-8a4de8477fd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:49.444361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7303b9-d281-4050-9a86-8a4de8477fd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:49.446858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7303b9-d281-4050-9a86-8a4de8477fd0 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:49.449997 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 524/1030] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:49.459027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413b32c-674e-4685-b0c4-859321ff17be None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:49.669437 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413b32c-674e-4685-b0c4-859321ff17be None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:49.726185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab86c92-2ff6-4561-8c3e-1ada363e5cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:49.726753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab86c92-2ff6-4561-8c3e-1ada363e5cae None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:49.727254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab86c92-2ff6-4561-8c3e-1ada363e5cae None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:49.727730 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab86c92-2ff6-4561-8c3e-1ada363e5cae None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:49.728948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab86c92-2ff6-4561-8c3e-1ada363e5cae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:49.731749 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab86c92-2ff6-4561-8c3e-1ada363e5cae None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:49.865888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 507/1031] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:10:49 2022] POST /identity/v3/users => generated 285 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:10:50.257514 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad4cdf-9e1b-49f8-a714-36c7a369778d None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:50.258276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad4cdf-9e1b-49f8-a714-36c7a369778d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:50.258276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad4cdf-9e1b-49f8-a714-36c7a369778d None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:50.260000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 525/1032] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:50.267837 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003675d-d48b-42f5-a2b2-55caecd3da5c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:50.268509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003675d-d48b-42f5-a2b2-55caecd3da5c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:50.269075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003675d-d48b-42f5-a2b2-55caecd3da5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:50.301195 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-4003675d-d48b-42f5-a2b2-55caecd3da5c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:50.303322 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-4003675d-d48b-42f5-a2b2-55caecd3da5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:50.401214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:50.437607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 508/1033] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:50 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:50.788279 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad4cdf-9e1b-49f8-a714-36c7a369778d None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:50.926865 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:50.993370 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad4cdf-9e1b-49f8-a714-36c7a369778d None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:51.070495 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687da582-3461-4967-8c23-bf8a764b82fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:51.071537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687da582-3461-4967-8c23-bf8a764b82fa None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:51.072009 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687da582-3461-4967-8c23-bf8a764b82fa None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:51.072481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687da582-3461-4967-8c23-bf8a764b82fa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:51.094366 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687da582-3461-4967-8c23-bf8a764b82fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:10:51.104436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-687da582-3461-4967-8c23-bf8a764b82fa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:10:51.104436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:51.104436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:51.104436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:51.104436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:51.104436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:51.104436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:51.104436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:51.104436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:51.104436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:51.104436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:51.104436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:51.104436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:10:51.104436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:10:51.104436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:10:51.104436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:10:51.105827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:10:51.105827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:10:51.105827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:10:51.105827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:10:51.105827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:51.105827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:51.105827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:51.105827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:51.105827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:51.105827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:51.105827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:51.105827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:51.105827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:51.105827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:51.105827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:51.105827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:51.107708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:51.107708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:51.107708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:51.107708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:51.107708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:10:51.107708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:10:51.107708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:10:51.107708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:10:51.107708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:10:51.107708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:10:51.107708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:10:51.107708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:51.111472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 526/1034] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Sep 7 13:10:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:51.116325 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003675d-d48b-42f5-a2b2-55caecd3da5c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:51.235512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67512e6-295e-461b-b900-e626bd89aac5 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:51.236039 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67512e6-295e-461b-b900-e626bd89aac5 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:51.236391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67512e6-295e-461b-b900-e626bd89aac5 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:51.239303 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 527/1035] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:10:51.248653 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c4495-1793-4d36-bce1-99b0a72a9283 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:51.249502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c4495-1793-4d36-bce1-99b0a72a9283 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:51.249502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c4495-1793-4d36-bce1-99b0a72a9283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:51.279404 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-bf8c4495-1793-4d36-bce1-99b0a72a9283 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:51.281248 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-bf8c4495-1793-4d36-bce1-99b0a72a9283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:51.284294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003675d-d48b-42f5-a2b2-55caecd3da5c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:51.360161 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05136a-811b-4ec9-9936-64ffc9f24713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:51.360837 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05136a-811b-4ec9-9936-64ffc9f24713 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:51.361251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05136a-811b-4ec9-9936-64ffc9f24713 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:51.361647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05136a-811b-4ec9-9936-64ffc9f24713 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:51.362886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05136a-811b-4ec9-9936-64ffc9f24713 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:51.363470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05136a-811b-4ec9-9936-64ffc9f24713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:51.366180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05136a-811b-4ec9-9936-64ffc9f24713 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:51.378762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 509/1036] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Sep 7 13:10:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:51.388569 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05136a-811b-4ec9-9936-64ffc9f24713 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:51.397886 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 528/1037] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:51 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:10:51.483761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:51.586196 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05136a-811b-4ec9-9936-64ffc9f24713 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:51.636925 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ad1f1-542a-498d-b90a-63329a8c61e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:51.637909 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ad1f1-542a-498d-b90a-63329a8c61e4 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:51.638872 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ad1f1-542a-498d-b90a-63329a8c61e4 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:51.639341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ad1f1-542a-498d-b90a-63329a8c61e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:51.660802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ad1f1-542a-498d-b90a-63329a8c61e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:51.663887 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ad1f1-542a-498d-b90a-63329a8c61e4 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:51.689614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-838ad1f1-542a-498d-b90a-63329a8c61e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:51.689614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:51.689614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:51.689614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:51.689614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:51.689614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:51.689614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:51.689614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:51.689614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:51.689614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:51.689614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:51.689614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:51.689614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:51.689614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:51.689614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:51.689614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:51.691567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:51.691567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:51.691567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:51.691567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:51.691567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:51.691567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:51.691567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:51.691567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:51.691567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:51.691567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:51.691567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:51.691567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:51.691567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:51.691567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:51.691567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:51.691567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:51.691567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:51.693494 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:51.693494 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:51.693494 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:51.695470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 510/1038] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:51.696081 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c4495-1793-4d36-bce1-99b0a72a9283 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:51.702279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ad1f1-542a-498d-b90a-63329a8c61e4 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:51.785038 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:51.840842 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c4495-1793-4d36-bce1-99b0a72a9283 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:51.885637 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:51.896924 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ad1f1-542a-498d-b90a-63329a8c61e4 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:51.911752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13819600-4b6a-4147-8bc6-0538017cccb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:51.912929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13819600-4b6a-4147-8bc6-0538017cccb9 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:51.913376 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13819600-4b6a-4147-8bc6-0538017cccb9 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:51.913938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13819600-4b6a-4147-8bc6-0538017cccb9 None admin] PATH_INFO: `/v3/services/image` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:51.914686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13819600-4b6a-4147-8bc6-0538017cccb9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:51.917120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13819600-4b6a-4147-8bc6-0538017cccb9 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:51.934388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-13819600-4b6a-4147-8bc6-0538017cccb9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:10:51.934388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:51.934388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:10:51.934388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:10:51.934388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:10:51.934388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:10:51.934388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:10:51.934388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:51.934388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:10:51.934388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:51.934388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:10:51.934388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:51.934388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:51.934388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:51.934388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:51.934388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:51.934388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:51.939225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:51.939225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:51.939225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:51.939225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:51.939225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:51.939225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:51.939225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:10:51.939225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:10:51.939225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:10:51.939225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:10:51.939225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:51.939225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:51.939225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:51.939225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:51.939225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:51.939225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:51.941139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:51.941139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:51.941139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:51.941139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:51.941139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:51.941139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:51.941139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:51.941139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:51.941139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:51.941139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:51.941139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:10:51.941139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:10:51.941139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 13:10:51.941139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:51.941139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 529/1039] 10.222.0.62 () {56 vars in 1105 bytes} [Wed Sep 7 13:10:51 2022] GET /identity/v3/services/image => generated 86 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:51.947314 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13819600-4b6a-4147-8bc6-0538017cccb9 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:51.959175 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d68fa4-f145-4113-b11c-d42cc81191b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:51.960023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d68fa4-f145-4113-b11c-d42cc81191b1 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:51.960333 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d68fa4-f145-4113-b11c-d42cc81191b1 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:51.960551 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d68fa4-f145-4113-b11c-d42cc81191b1 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:51.961238 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d68fa4-f145-4113-b11c-d42cc81191b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:51.961548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d68fa4-f145-4113-b11c-d42cc81191b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:51.963027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d68fa4-f145-4113-b11c-d42cc81191b1 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:51.981212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 511/1040] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:51.990175 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d68fa4-f145-4113-b11c-d42cc81191b1 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:52.101640 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13819600-4b6a-4147-8bc6-0538017cccb9 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:52.148275 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:52.161569 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd863aea-ff37-4e6c-bd6d-a924f87ba363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:52.162761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd863aea-ff37-4e6c-bd6d-a924f87ba363 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:52.163205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd863aea-ff37-4e6c-bd6d-a924f87ba363 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:52.164792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd863aea-ff37-4e6c-bd6d-a924f87ba363 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:52.164792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd863aea-ff37-4e6c-bd6d-a924f87ba363 None admin] RBAC: Adding query filter params (name=image) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:52.164792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd863aea-ff37-4e6c-bd6d-a924f87ba363 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:52.166038 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d68fa4-f145-4113-b11c-d42cc81191b1 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:52.167047 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd863aea-ff37-4e6c-bd6d-a924f87ba363 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:52.180667 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 530/1041] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:52.182610 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd863aea-ff37-4e6c-bd6d-a924f87ba363 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:52.220288 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457e689-ac1b-41af-866d-9b06878ff0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:52.220989 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457e689-ac1b-41af-866d-9b06878ff0cd None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:52.221355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457e689-ac1b-41af-866d-9b06878ff0cd None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:52.221649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457e689-ac1b-41af-866d-9b06878ff0cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:52.231545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d457e689-ac1b-41af-866d-9b06878ff0cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:52.233426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d457e689-ac1b-41af-866d-9b06878ff0cd None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:52.250034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-d457e689-ac1b-41af-866d-9b06878ff0cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:10:52.250034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:52.250034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:52.250034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:52.250034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:52.250034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:52.250034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:52.250034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:52.250034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:52.250034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:52.250034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:52.250034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:52.250034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:10:52.250034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:10:52.250034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:10:52.250034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:52.252221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:52.253941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:52.253941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:52.253941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:10:52.253941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:10:52.253941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:10:52.253941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:10:52.253941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:10:52.253941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:10:52.253941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:10:52.253941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:52.257185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 512/1042] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 7 13:10:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:52.258202 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457e689-ac1b-41af-866d-9b06878ff0cd None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:52.282863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:52.353130 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd863aea-ff37-4e6c-bd6d-a924f87ba363 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:52.425808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e432b-1f57-40a4-9296-1e53beea11a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:52.426658 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e432b-1f57-40a4-9296-1e53beea11a8 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:52.427203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e432b-1f57-40a4-9296-1e53beea11a8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:52.427672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e432b-1f57-40a4-9296-1e53beea11a8 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:52.428801 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e432b-1f57-40a4-9296-1e53beea11a8 None admin] RBAC: Adding query filter params (type=image) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:52.429393 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e432b-1f57-40a4-9296-1e53beea11a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:52.432209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e432b-1f57-40a4-9296-1e53beea11a8 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:52.443033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 531/1043] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:52.453259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e432b-1f57-40a4-9296-1e53beea11a8 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:52.473133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457e689-ac1b-41af-866d-9b06878ff0cd None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:52.549006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061e648-0797-4a0d-8053-a189a0ae916b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:52.550096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061e648-0797-4a0d-8053-a189a0ae916b None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:52.550582 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061e648-0797-4a0d-8053-a189a0ae916b None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:52.551085 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061e648-0797-4a0d-8053-a189a0ae916b None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:52.552544 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061e648-0797-4a0d-8053-a189a0ae916b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:52.553040 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061e648-0797-4a0d-8053-a189a0ae916b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:52.555952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061e648-0797-4a0d-8053-a189a0ae916b None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:52.572678 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:52.594038 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 513/1044] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 7 13:10:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:52.601054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061e648-0797-4a0d-8053-a189a0ae916b None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:52.664353 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e432b-1f57-40a4-9296-1e53beea11a8 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:52.736366 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16598b-c634-4dae-bf81-8651c39d3ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:52.737834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16598b-c634-4dae-bf81-8651c39d3ffb None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:52.738670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16598b-c634-4dae-bf81-8651c39d3ffb None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:52.739417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16598b-c634-4dae-bf81-8651c39d3ffb None admin] PATH_INFO: `/v3/endpoints` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:52.740495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:52.740683 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16598b-c634-4dae-bf81-8651c39d3ffb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:52.743569 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16598b-c634-4dae-bf81-8651c39d3ffb None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:52.792404 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061e648-0797-4a0d-8053-a189a0ae916b None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:52.851402 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c100d-012c-4f6c-a0e3-dcdecb81b110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:52.852241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c100d-012c-4f6c-a0e3-dcdecb81b110 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:52.852610 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c100d-012c-4f6c-a0e3-dcdecb81b110 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:52.853017 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c100d-012c-4f6c-a0e3-dcdecb81b110 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:52.879717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 532/1045] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:10:52 2022] POST /identity/v3/endpoints => generated 327 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:10:53.010489 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c100d-012c-4f6c-a0e3-dcdecb81b110 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:53.013319 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c100d-012c-4f6c-a0e3-dcdecb81b110 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:53.160743 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:53.219585 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 514/1046] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:10:52 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:10:53.230427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16598b-c634-4dae-bf81-8651c39d3ffb None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:53.306113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2751ad-d7e4-459d-9737-d3679f6d8fee None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:53.306421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2751ad-d7e4-459d-9737-d3679f6d8fee None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:53.306745 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2751ad-d7e4-459d-9737-d3679f6d8fee None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:53.308355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 515/1047] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:10:53.318887 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddaae2-b93c-4ffb-ac81-bdfa65005cbc None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:53.319383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddaae2-b93c-4ffb-ac81-bdfa65005cbc None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:53.319760 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ddaae2-b93c-4ffb-ac81-bdfa65005cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:53.344342 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-d5ddaae2-b93c-4ffb-ac81-bdfa65005cbc None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:53.345672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d5ddaae2-b93c-4ffb-ac81-bdfa65005cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:53.417405 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16598b-c634-4dae-bf81-8651c39d3ffb None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:53.432209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:53.472226 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 516/1048] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:10:53.483884 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dee896-e39f-4b6c-9e21-c341d091252c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:53.484592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dee896-e39f-4b6c-9e21-c341d091252c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:53.484900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dee896-e39f-4b6c-9e21-c341d091252c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:53.485361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dee896-e39f-4b6c-9e21-c341d091252c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:53.498245 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dee896-e39f-4b6c-9e21-c341d091252c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:53.500628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dee896-e39f-4b6c-9e21-c341d091252c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:53.522710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-c8dee896-e39f-4b6c-9e21-c341d091252c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:53.522710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:53.522710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:53.522710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:53.522710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:53.522710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:53.522710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:53.522710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:53.522710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:53.522710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:53.522710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:53.522710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:53.522710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:53.522710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:53.522710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:53.522710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:53.524135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:53.524135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:53.524135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:53.524135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:53.524135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:53.524135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:53.524135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:53.524135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:53.524135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:53.524135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:53.524135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:53.524135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:53.524135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:53.524135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:53.524135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:53.524135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:53.524135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:53.527269 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:53.527269 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:53.527269 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:53.532356 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 533/1049] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:53.535809 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ddaae2-b93c-4ffb-ac81-bdfa65005cbc None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:53.718509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ddaae2-b93c-4ffb-ac81-bdfa65005cbc None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:53.721040 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:53.783848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f042bcc-2afa-4376-b9cc-e8f8e0255a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:53.785076 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f042bcc-2afa-4376-b9cc-e8f8e0255a2c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:53.785448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f042bcc-2afa-4376-b9cc-e8f8e0255a2c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:53.785880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f042bcc-2afa-4376-b9cc-e8f8e0255a2c None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:53.787129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f042bcc-2afa-4376-b9cc-e8f8e0255a2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:53.787807 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f042bcc-2afa-4376-b9cc-e8f8e0255a2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:53.790522 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f042bcc-2afa-4376-b9cc-e8f8e0255a2c None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:53.817055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 517/1050] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:53.820858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dee896-e39f-4b6c-9e21-c341d091252c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:53.847140 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f042bcc-2afa-4376-b9cc-e8f8e0255a2c None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:54.014451 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f042bcc-2afa-4376-b9cc-e8f8e0255a2c None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:54.019979 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dee896-e39f-4b6c-9e21-c341d091252c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:54.063823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0416c-8eee-4bcb-8def-c9c494e6420c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:54.064794 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0416c-8eee-4bcb-8def-c9c494e6420c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:54.065057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0416c-8eee-4bcb-8def-c9c494e6420c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:54.065335 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0416c-8eee-4bcb-8def-c9c494e6420c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:54.069755 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e3d64-21be-4dcf-9f61-e7d3c5dc60e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:54.070403 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e3d64-21be-4dcf-9f61-e7d3c5dc60e8 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:54.070740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e3d64-21be-4dcf-9f61-e7d3c5dc60e8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:54.071106 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e3d64-21be-4dcf-9f61-e7d3c5dc60e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:54.080250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0416c-8eee-4bcb-8def-c9c494e6420c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:10:54.080476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e3d64-21be-4dcf-9f61-e7d3c5dc60e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:54.081841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e3d64-21be-4dcf-9f61-e7d3c5dc60e8 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:54.086227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-b0d0416c-8eee-4bcb-8def-c9c494e6420c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:10:54.086227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:54.086227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:54.086227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:54.086227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:54.086227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:54.086227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:54.086227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:54.086227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:54.086227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:54.086227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:54.086227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:54.086227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:10:54.086227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:10:54.086227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:10:54.086227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:10:54.087947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:10:54.087947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:10:54.087947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:10:54.087947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:10:54.087947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:54.087947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:54.087947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:54.087947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:54.087947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:54.087947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:54.087947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:54.087947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:54.087947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:54.087947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:54.087947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:54.087947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:54.088813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:54.088813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:54.088813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:54.088813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:54.088813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:10:54.088813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:10:54.088813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:10:54.088813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:10:54.088813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:10:54.088813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:10:54.088813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:10:54.088813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:54.091445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 518/1051] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:10:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:54.094371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0416c-8eee-4bcb-8def-c9c494e6420c None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:54.098671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-cd1e3d64-21be-4dcf-9f61-e7d3c5dc60e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:54.098671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:54.098671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:54.098671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:54.098671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:54.098671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:54.098671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:54.098671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:54.098671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:54.098671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:54.098671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:54.098671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:54.098671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:10:54.098671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:10:54.098671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:10:54.100291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:10:54.100291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:54.100291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:54.100291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:54.100291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:54.100291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:54.100291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:54.100291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:54.100291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:54.100291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:54.100291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:54.100291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:54.100291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:54.100291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:54.100291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:54.100291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:54.100291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:54.101239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:10:54.101239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:10:54.101239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:10:54.101239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:10:54.101239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:10:54.101239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:10:54.101239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:10:54.101239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:54.106760 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 534/1052] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:10:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:54.108240 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e3d64-21be-4dcf-9f61-e7d3c5dc60e8 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:54.129517 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:54.132618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:54.250394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0416c-8eee-4bcb-8def-c9c494e6420c None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:54.283750 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e3d64-21be-4dcf-9f61-e7d3c5dc60e8 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:54.311229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f8572-bd41-463c-92d6-6ad775976b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:54.312333 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f8572-bd41-463c-92d6-6ad775976b5e None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:54.312598 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f8572-bd41-463c-92d6-6ad775976b5e None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:54.313190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f8572-bd41-463c-92d6-6ad775976b5e None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:54.314435 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f8572-bd41-463c-92d6-6ad775976b5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:54.314989 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f8572-bd41-463c-92d6-6ad775976b5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:54.317985 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f8572-bd41-463c-92d6-6ad775976b5e None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:54.335871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f07a05-59ec-4b11-80ce-66b29f234c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:54.336396 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f07a05-59ec-4b11-80ce-66b29f234c9e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:54.336671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f07a05-59ec-4b11-80ce-66b29f234c9e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:54.336981 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f07a05-59ec-4b11-80ce-66b29f234c9e None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:54.337616 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 519/1053] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:10:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:54.337785 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f07a05-59ec-4b11-80ce-66b29f234c9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:54.337950 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f07a05-59ec-4b11-80ce-66b29f234c9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:54.339434 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f07a05-59ec-4b11-80ce-66b29f234c9e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:54.345303 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f8572-bd41-463c-92d6-6ad775976b5e None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:54.355051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 535/1054] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:10:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:54.361447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f07a05-59ec-4b11-80ce-66b29f234c9e None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:54.475094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:54.493123 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:54.499335 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f8572-bd41-463c-92d6-6ad775976b5e None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:54.528014 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f07a05-59ec-4b11-80ce-66b29f234c9e None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:54.557579 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bc02d-53ba-42de-8f28-13a6ebc9e0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:54.558278 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bc02d-53ba-42de-8f28-13a6ebc9e0e6 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:54.558737 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bc02d-53ba-42de-8f28-13a6ebc9e0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:54.559259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bc02d-53ba-42de-8f28-13a6ebc9e0e6 None admin] PATH_INFO: `/v3/users/66dde655d0e74efab8029d3cd47e0129` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:54.581964 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bc02d-53ba-42de-8f28-13a6ebc9e0e6 None admin] RBAC: Authorizing `identity:get_user(user_id=66dde655d0e74efab8029d3cd47e0129)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:54.583931 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bc02d-53ba-42de-8f28-13a6ebc9e0e6 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:54.586654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 520/1055] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:10:54 2022] GET /identity/v3/users/66dde655d0e74efab8029d3cd47e0129 => generated 285 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:54.596368 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bc02d-53ba-42de-8f28-13a6ebc9e0e6 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:54.600479 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724da16-1a8e-4607-b1d9-5b2506df8356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:54.601131 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724da16-1a8e-4607-b1d9-5b2506df8356 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:54.601408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724da16-1a8e-4607-b1d9-5b2506df8356 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:54.601704 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724da16-1a8e-4607-b1d9-5b2506df8356 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:54.602467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724da16-1a8e-4607-b1d9-5b2506df8356 None admin] RBAC: Adding query filter params (role.id=e88e2f39fcb94166851f6db7b9f9dfcb, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=97c363046db444cbb3d4346066f9ad42) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:54.602772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724da16-1a8e-4607-b1d9-5b2506df8356 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:54.604279 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724da16-1a8e-4607-b1d9-5b2506df8356 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:54.612523 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 536/1056] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:10:54 2022] GET /identity/v3/role_assignments?user.id=97c363046db444cbb3d4346066f9ad42&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=e88e2f39fcb94166851f6db7b9f9dfcb => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:54.799120 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bc02d-53ba-42de-8f28-13a6ebc9e0e6 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:54.877145 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4fa0b-0148-4331-a3dc-a6e53b25999f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:54.878162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4fa0b-0148-4331-a3dc-a6e53b25999f None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:54.878670 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4fa0b-0148-4331-a3dc-a6e53b25999f None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:54.879213 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4fa0b-0148-4331-a3dc-a6e53b25999f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:54.909440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:55.028724 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4fa0b-0148-4331-a3dc-a6e53b25999f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:55.032003 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4fa0b-0148-4331-a3dc-a6e53b25999f None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:55.244042 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 521/1057] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:10:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 13:10:55.255539 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724da16-1a8e-4607-b1d9-5b2506df8356 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:55.456264 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724da16-1a8e-4607-b1d9-5b2506df8356 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:55.520271 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e41b8-60b6-4d66-bf61-4e69a54dc3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:55.522587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e41b8-60b6-4d66-bf61-4e69a54dc3a7 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:55.523420 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e41b8-60b6-4d66-bf61-4e69a54dc3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:55.524206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e41b8-60b6-4d66-bf61-4e69a54dc3a7 None admin] PATH_INFO: `/v3/projects/7795a64328534f71b54602bddd5c0c37` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:55.532923 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e41b8-60b6-4d66-bf61-4e69a54dc3a7 None admin] RBAC: Authorizing `identity:get_project(project_id=7795a64328534f71b54602bddd5c0c37)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:55.537755 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e41b8-60b6-4d66-bf61-4e69a54dc3a7 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:55.538727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 537/1058] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:10:55 2022] GET /identity/v3/projects/7795a64328534f71b54602bddd5c0c37 => generated 311 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:55.549639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4fa0b-0148-4331-a3dc-a6e53b25999f None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:55.627556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37044f83-e2ef-4b01-af06-57c31078a974 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:55.628144 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37044f83-e2ef-4b01-af06-57c31078a974 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:55.628611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37044f83-e2ef-4b01-af06-57c31078a974 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:55.631226 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 538/1059] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:10:55.640874 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aee2ba-f2ae-4c11-838d-87a88147161c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:55.641345 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aee2ba-f2ae-4c11-838d-87a88147161c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:55.641827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aee2ba-f2ae-4c11-838d-87a88147161c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:55.643306 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:55.676008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b5aee2ba-f2ae-4c11-838d-87a88147161c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:55.678220 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b5aee2ba-f2ae-4c11-838d-87a88147161c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:55.683848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:55.761997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4fa0b-0148-4331-a3dc-a6e53b25999f None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:55.809108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 539/1060] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:55.832059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6a7f5-3585-4de3-8b1d-320c7af4f81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:55.833007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6a7f5-3585-4de3-8b1d-320c7af4f81c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:55.833373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6a7f5-3585-4de3-8b1d-320c7af4f81c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:55.833791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6a7f5-3585-4de3-8b1d-320c7af4f81c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:55.834692 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6a7f5-3585-4de3-8b1d-320c7af4f81c None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=7795a64328534f71b54602bddd5c0c37, user.id=66dde655d0e74efab8029d3cd47e0129) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:55.835186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6a7f5-3585-4de3-8b1d-320c7af4f81c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:55.836859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6a7f5-3585-4de3-8b1d-320c7af4f81c None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:55.855093 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 522/1061] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:10:55 2022] GET /identity/v3/role_assignments?user.id=66dde655d0e74efab8029d3cd47e0129&scope.project.id=7795a64328534f71b54602bddd5c0c37&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 259 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:56.331622 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aee2ba-f2ae-4c11-838d-87a88147161c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:56.486745 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aee2ba-f2ae-4c11-838d-87a88147161c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:56.543354 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30d4d1-9914-455f-963a-2d7aeb0f828d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:56.544388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30d4d1-9914-455f-963a-2d7aeb0f828d None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:56.545626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30d4d1-9914-455f-963a-2d7aeb0f828d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:56.546136 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30d4d1-9914-455f-963a-2d7aeb0f828d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:56.567796 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:56.692586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30d4d1-9914-455f-963a-2d7aeb0f828d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:56.695428 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30d4d1-9914-455f-963a-2d7aeb0f828d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:56.913406 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 540/1062] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:10:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:10:56.924339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6a7f5-3585-4de3-8b1d-320c7af4f81c None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:57.131330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6a7f5-3585-4de3-8b1d-320c7af4f81c None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:57.146049 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:57.215927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27822259-813e-49f4-9c8f-33e8468c7581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:57.218338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27822259-813e-49f4-9c8f-33e8468c7581 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:57.219208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27822259-813e-49f4-9c8f-33e8468c7581 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:57.219938 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27822259-813e-49f4-9c8f-33e8468c7581 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:57.243565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27822259-813e-49f4-9c8f-33e8468c7581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:57.245874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27822259-813e-49f4-9c8f-33e8468c7581 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:57.273526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-27822259-813e-49f4-9c8f-33e8468c7581 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:57.273526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:57.273526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:57.273526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:57.273526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:57.273526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:57.273526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:57.273526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:57.273526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:57.273526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:57.273526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:57.273526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:57.273526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:57.273526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:57.273526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:57.273526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:57.275999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:57.275999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:57.275999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:57.275999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:57.275999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:57.275999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:57.275999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:57.275999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:57.275999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:57.275999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:57.275999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:57.275999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:57.275999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:57.275999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:57.275999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:57.275999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:57.275999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:57.277837 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:57.277837 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:57.277837 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:57.278911 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 523/1063] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:57.286587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30d4d1-9914-455f-963a-2d7aeb0f828d None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:57.367189 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:57.485258 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30d4d1-9914-455f-963a-2d7aeb0f828d None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:57.530315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546adfb9-02e8-4079-87c2-260fc24322bf None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:57.530315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546adfb9-02e8-4079-87c2-260fc24322bf None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:57.530315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546adfb9-02e8-4079-87c2-260fc24322bf None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:57.530315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 524/1064] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:10:57.537862 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19bb6b-9b83-405b-b465-77f77797518d None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:57.538339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19bb6b-9b83-405b-b465-77f77797518d None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:57.538807 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19bb6b-9b83-405b-b465-77f77797518d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:57.553128 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cabd9d-95b6-4956-8c5d-a9ca41405b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:57.553874 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cabd9d-95b6-4956-8c5d-a9ca41405b62 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:57.554131 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cabd9d-95b6-4956-8c5d-a9ca41405b62 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:57.554356 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cabd9d-95b6-4956-8c5d-a9ca41405b62 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:57.556711 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cabd9d-95b6-4956-8c5d-a9ca41405b62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:57.556711 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cabd9d-95b6-4956-8c5d-a9ca41405b62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:57.558774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cabd9d-95b6-4956-8c5d-a9ca41405b62 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:57.569676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-ec19bb6b-9b83-405b-b465-77f77797518d None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:57.571962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ec19bb6b-9b83-405b-b465-77f77797518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:57.579936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 541/1065] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:57.682751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 525/1066] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:10:57.989945 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cabd9d-95b6-4956-8c5d-a9ca41405b62 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:58.135542 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:58.140039 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cabd9d-95b6-4956-8c5d-a9ca41405b62 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:58.183599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece62ff-cdaf-499b-b14e-4ef795fed8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:58.184337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece62ff-cdaf-499b-b14e-4ef795fed8c2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:58.184674 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece62ff-cdaf-499b-b14e-4ef795fed8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:58.185008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece62ff-cdaf-499b-b14e-4ef795fed8c2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:58.199229 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece62ff-cdaf-499b-b14e-4ef795fed8c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:10:58.205181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-4ece62ff-cdaf-499b-b14e-4ef795fed8c2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:10:58.205181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:58.205181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:58.205181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:58.205181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:58.205181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:58.205181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:58.205181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:58.205181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:58.205181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:58.205181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:58.205181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:58.205181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:10:58.205181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:10:58.205181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:10:58.205181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:10:58.207331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:10:58.207331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:10:58.207331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:10:58.207331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:10:58.207331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:58.207331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:58.207331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:58.207331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:58.207331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:58.207331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:58.207331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:58.207331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:58.207331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:58.207331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:58.207331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:58.207331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:58.209285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:58.209285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:58.209285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:58.209285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:58.209285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:10:58.209285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:10:58.209285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:10:58.209285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:10:58.209285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:10:58.209285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:10:58.209285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:10:58.209285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:58.211212 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 542/1067] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Sep 7 13:10:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:58.215247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19bb6b-9b83-405b-b465-77f77797518d None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:58.370661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:58.385512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19bb6b-9b83-405b-b465-77f77797518d None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:58.428806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8978d40-8548-4399-bb51-d9c5c5b10f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:58.429516 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8978d40-8548-4399-bb51-d9c5c5b10f8e None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:58.429841 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8978d40-8548-4399-bb51-d9c5c5b10f8e None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:58.430112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8978d40-8548-4399-bb51-d9c5c5b10f8e None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:58.430916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8978d40-8548-4399-bb51-d9c5c5b10f8e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:58.431252 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8978d40-8548-4399-bb51-d9c5c5b10f8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:58.432665 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8978d40-8548-4399-bb51-d9c5c5b10f8e None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:58.444017 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 526/1068] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Sep 7 13:10:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:58.451531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece62ff-cdaf-499b-b14e-4ef795fed8c2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:58.613365 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece62ff-cdaf-499b-b14e-4ef795fed8c2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:58.634298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26856c-dc21-453d-84e6-6396217ecf02 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:58.634519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26856c-dc21-453d-84e6-6396217ecf02 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:58.634749 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26856c-dc21-453d-84e6-6396217ecf02 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:58.636130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 527/1069] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:10:58.644980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff366ebc-ad31-49bd-ac61-0925da737798 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:58.645606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff366ebc-ad31-49bd-ac61-0925da737798 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:58.645951 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff366ebc-ad31-49bd-ac61-0925da737798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:58.656255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8649821-d06f-409f-b1fd-cf8f14a5c711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:58.657112 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8649821-d06f-409f-b1fd-cf8f14a5c711 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:58.657477 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8649821-d06f-409f-b1fd-cf8f14a5c711 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:58.657838 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8649821-d06f-409f-b1fd-cf8f14a5c711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:58.667482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-ff366ebc-ad31-49bd-ac61-0925da737798 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:10:58.668751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ff366ebc-ad31-49bd-ac61-0925da737798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:10:58.672336 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8649821-d06f-409f-b1fd-cf8f14a5c711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:58.673896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8649821-d06f-409f-b1fd-cf8f14a5c711 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:58.696101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-e8649821-d06f-409f-b1fd-cf8f14a5c711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:58.696101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:58.696101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:58.696101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:58.696101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:58.696101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:58.696101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:58.696101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:58.696101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:58.696101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:58.696101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:58.696101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:58.696101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:58.696101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:58.696101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:58.696101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:58.698617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:58.698617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:58.698617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:58.698617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:58.698617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:58.698617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:58.698617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:58.698617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:58.698617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:58.698617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:58.698617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:58.698617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:58.698617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:58.698617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:58.698617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:58.698617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:58.698617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:58.701401 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:58.701401 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:58.701401 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:58.701401 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 543/1070] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:58.704992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8649821-d06f-409f-b1fd-cf8f14a5c711 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:58.725324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:58.784108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 528/1071] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:10:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:10:58.866663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8649821-d06f-409f-b1fd-cf8f14a5c711 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:58.912883 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3579080-8dc9-424d-ae7c-31c44bd62808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:58.913438 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3579080-8dc9-424d-ae7c-31c44bd62808 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:58.913715 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3579080-8dc9-424d-ae7c-31c44bd62808 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:58.914096 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3579080-8dc9-424d-ae7c-31c44bd62808 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:58.914820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3579080-8dc9-424d-ae7c-31c44bd62808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:58.915296 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3579080-8dc9-424d-ae7c-31c44bd62808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:58.916889 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3579080-8dc9-424d-ae7c-31c44bd62808 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:58.941309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 544/1072] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:58.947232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff366ebc-ad31-49bd-ac61-0925da737798 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:59.046150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:59.068834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff366ebc-ad31-49bd-ac61-0925da737798 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:59.107997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122bf17-fc1c-4be6-a7ed-b87bb08d006e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:59.108639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122bf17-fc1c-4be6-a7ed-b87bb08d006e None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:59.108963 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122bf17-fc1c-4be6-a7ed-b87bb08d006e None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:59.109197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122bf17-fc1c-4be6-a7ed-b87bb08d006e None admin] PATH_INFO: `/v3/users/nova` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:59.117113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122bf17-fc1c-4be6-a7ed-b87bb08d006e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:59.118568 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d122bf17-fc1c-4be6-a7ed-b87bb08d006e None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:59.131270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-d122bf17-fc1c-4be6-a7ed-b87bb08d006e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:10:59.131270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:59.131270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:59.131270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:59.131270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:59.131270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:59.131270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:59.131270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:59.131270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:59.131270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:59.131270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:59.131270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:59.131270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:10:59.131270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:10:59.131270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:10:59.131270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:59.133051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:59.135133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:59.135133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:59.135133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:10:59.135133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:10:59.135133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:10:59.135133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:10:59.135133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:10:59.135133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:10:59.135133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:10:59.135133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:59.135133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 529/1073] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 7 13:10:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:59.138913 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3579080-8dc9-424d-ae7c-31c44bd62808 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:59.193472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122bf17-fc1c-4be6-a7ed-b87bb08d006e None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:59.229722 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:59.278820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3579080-8dc9-424d-ae7c-31c44bd62808 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:59.324018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb9ac8-8545-45e0-abb2-f59ec3c61d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:59.325083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb9ac8-8545-45e0-abb2-f59ec3c61d6a None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:59.325562 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb9ac8-8545-45e0-abb2-f59ec3c61d6a None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:59.326577 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb9ac8-8545-45e0-abb2-f59ec3c61d6a None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:59.327974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb9ac8-8545-45e0-abb2-f59ec3c61d6a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:59.328541 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb9ac8-8545-45e0-abb2-f59ec3c61d6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:59.331595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb9ac8-8545-45e0-abb2-f59ec3c61d6a None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:59.354213 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122bf17-fc1c-4be6-a7ed-b87bb08d006e None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:59.359484 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 545/1074] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 7 13:10:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:10:59.365525 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb9ac8-8545-45e0-abb2-f59ec3c61d6a None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:59.405321 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff1125-98e4-45bb-9c55-21176831340d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:59.405966 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff1125-98e4-45bb-9c55-21176831340d None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:59.406249 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff1125-98e4-45bb-9c55-21176831340d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:59.406529 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff1125-98e4-45bb-9c55-21176831340d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:59.415084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:59.418881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff1125-98e4-45bb-9c55-21176831340d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:10:59.423945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-45ff1125-98e4-45bb-9c55-21176831340d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:10:59.423945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:59.423945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:59.423945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:59.423945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:59.423945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:59.423945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:59.423945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:59.423945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:59.423945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:59.423945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:59.423945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:59.423945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:10:59.423945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:10:59.423945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:10:59.423945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:10:59.425391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:10:59.425391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:10:59.425391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:10:59.425391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:10:59.425391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:59.425391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:59.425391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:59.425391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:59.425391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:59.425391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:59.425391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:59.425391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:59.425391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:59.425391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:59.425391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:59.425391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:59.426440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:59.426440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:59.426440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:59.426440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:59.426440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:10:59.426440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:10:59.426440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:10:59.426440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:10:59.426440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:10:59.426440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:10:59.426440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:10:59.426440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:10:59.426440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 530/1075] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:10:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:10:59.433128 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff1125-98e4-45bb-9c55-21176831340d None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:59.533196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb9ac8-8545-45e0-abb2-f59ec3c61d6a None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:59.552016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff1125-98e4-45bb-9c55-21176831340d None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:59.575477 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:59.584187 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9fe44-2f59-4bdd-a132-a1f3cb4e0d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:59.584848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9fe44-2f59-4bdd-a132-a1f3cb4e0d8f None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:59.585166 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9fe44-2f59-4bdd-a132-a1f3cb4e0d8f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:59.585441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9fe44-2f59-4bdd-a132-a1f3cb4e0d8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:59.598802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9fe44-2f59-4bdd-a132-a1f3cb4e0d8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:59.600784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9fe44-2f59-4bdd-a132-a1f3cb4e0d8f None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:59.614895 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54e5d0-9df5-49ab-ac34-8ae4df5377d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:59.615941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54e5d0-9df5-49ab-ac34-8ae4df5377d8 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:59.616422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54e5d0-9df5-49ab-ac34-8ae4df5377d8 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:59.616987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54e5d0-9df5-49ab-ac34-8ae4df5377d8 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:59.618184 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54e5d0-9df5-49ab-ac34-8ae4df5377d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:59.618714 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54e5d0-9df5-49ab-ac34-8ae4df5377d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:59.618876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-a9c9fe44-2f59-4bdd-a132-a1f3cb4e0d8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:59.618876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:10:59.618876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:10:59.618876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:10:59.618876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:10:59.618876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:10:59.618876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:10:59.618876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:10:59.618876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:10:59.618876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:10:59.618876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:10:59.618876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:10:59.618876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:10:59.618876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:10:59.618876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:10:59.618876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:10:59.620871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:10:59.620871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:10:59.620871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:10:59.620871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:10:59.620871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:10:59.620871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:10:59.620871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:10:59.620871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:10:59.620871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:10:59.620871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:10:59.620871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:10:59.620871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:10:59.620871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:10:59.620871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:10:59.620871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:10:59.620871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:10:59.620871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:10:59.622066 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:10:59.622066 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:10:59.622066 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:10:59.622066 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 546/1076] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:10:59.622343 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54e5d0-9df5-49ab-ac34-8ae4df5377d8 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:59.627251 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9fe44-2f59-4bdd-a132-a1f3cb4e0d8f None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:59.637931 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 531/1077] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:10:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:59.647641 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54e5d0-9df5-49ab-ac34-8ae4df5377d8 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:59.745961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:59.764751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9fe44-2f59-4bdd-a132-a1f3cb4e0d8f None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:59.803154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54e5d0-9df5-49ab-ac34-8ae4df5377d8 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:10:59.808624 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fccc28-cdf3-4fcd-8ad7-7a48fd640dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:59.809351 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fccc28-cdf3-4fcd-8ad7-7a48fd640dc0 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:59.809631 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fccc28-cdf3-4fcd-8ad7-7a48fd640dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:59.809904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fccc28-cdf3-4fcd-8ad7-7a48fd640dc0 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:59.811162 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fccc28-cdf3-4fcd-8ad7-7a48fd640dc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:10:59.811468 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fccc28-cdf3-4fcd-8ad7-7a48fd640dc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:59.812890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fccc28-cdf3-4fcd-8ad7-7a48fd640dc0 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:59.830085 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 547/1078] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:59.836366 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fccc28-cdf3-4fcd-8ad7-7a48fd640dc0 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:59.867998 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f70e7b-418b-40ff-8120-f48c46d92d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:10:59.868934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f70e7b-418b-40ff-8120-f48c46d92d18 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:10:59.869324 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f70e7b-418b-40ff-8120-f48c46d92d18 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:10:59.869782 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f70e7b-418b-40ff-8120-f48c46d92d18 None admin] PATH_INFO: `/v3/users/66dde655d0e74efab8029d3cd47e0129` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:10:59.871269 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f70e7b-418b-40ff-8120-f48c46d92d18 None admin] RBAC: Authorizing `identity:get_user(user_id=66dde655d0e74efab8029d3cd47e0129)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:10:59.874404 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f70e7b-418b-40ff-8120-f48c46d92d18 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:10:59.876333 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 532/1079] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:10:59 2022] GET /identity/v3/users/66dde655d0e74efab8029d3cd47e0129 => generated 285 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:10:59.883228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f70e7b-418b-40ff-8120-f48c46d92d18 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:10:59.892839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:10:59.964364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fccc28-cdf3-4fcd-8ad7-7a48fd640dc0 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:00.025874 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b397a15-1df4-4d59-9c08-8c1c3fa96803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:00.026785 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b397a15-1df4-4d59-9c08-8c1c3fa96803 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:00.027202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b397a15-1df4-4d59-9c08-8c1c3fa96803 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:00.027593 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b397a15-1df4-4d59-9c08-8c1c3fa96803 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:00.036355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b397a15-1df4-4d59-9c08-8c1c3fa96803 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:00.038077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b397a15-1df4-4d59-9c08-8c1c3fa96803 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:00.042074 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f70e7b-418b-40ff-8120-f48c46d92d18 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:00.056602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-8b397a15-1df4-4d59-9c08-8c1c3fa96803 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:11:00.056602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:11:00.056602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:11:00.056602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:11:00.056602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:11:00.056602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:11:00.056602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:11:00.056602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:11:00.056602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:11:00.056602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:11:00.056602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:11:00.056602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:11:00.056602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:11:00.056602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:11:00.056602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:11:00.058357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:11:00.058357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:11:00.058357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:11:00.058357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:11:00.058357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:11:00.058357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:11:00.058357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:11:00.058357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:11:00.058357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:11:00.058357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:11:00.058357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:11:00.058357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:11:00.058357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:11:00.058357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:11:00.058357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:11:00.058357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:11:00.058357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:11:00.060029 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:11:00.060029 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:11:00.060029 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:11:00.060029 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:11:00.060029 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:11:00.060029 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:11:00.060029 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:11:00.060029 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:11:00.063179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 548/1080] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:10:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:11:00.064992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b397a15-1df4-4d59-9c08-8c1c3fa96803 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:00.101272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:00.108699 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76d6aa-69fc-4c7b-b123-1e7641cbec97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:00.109809 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76d6aa-69fc-4c7b-b123-1e7641cbec97 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:00.110106 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76d6aa-69fc-4c7b-b123-1e7641cbec97 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:00.115344 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76d6aa-69fc-4c7b-b123-1e7641cbec97 None admin] PATH_INFO: `/v3/projects/7795a64328534f71b54602bddd5c0c37` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:00.115344 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76d6aa-69fc-4c7b-b123-1e7641cbec97 None admin] RBAC: Authorizing `identity:get_project(project_id=7795a64328534f71b54602bddd5c0c37)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:00.115344 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76d6aa-69fc-4c7b-b123-1e7641cbec97 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:00.116141 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 533/1081] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:10:59 2022] GET /identity/v3/projects/7795a64328534f71b54602bddd5c0c37 => generated 311 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:11:00.124015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76d6aa-69fc-4c7b-b123-1e7641cbec97 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:00.225052 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b397a15-1df4-4d59-9c08-8c1c3fa96803 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:00.228147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:00.285158 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b4f85-628f-4ec4-8e29-24f43bf5df33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:00.285686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b4f85-628f-4ec4-8e29-24f43bf5df33 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:00.285961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b4f85-628f-4ec4-8e29-24f43bf5df33 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:00.286229 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b4f85-628f-4ec4-8e29-24f43bf5df33 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:00.287084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b4f85-628f-4ec4-8e29-24f43bf5df33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:00.287396 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b4f85-628f-4ec4-8e29-24f43bf5df33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:00.288950 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b4f85-628f-4ec4-8e29-24f43bf5df33 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:00.300384 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76d6aa-69fc-4c7b-b123-1e7641cbec97 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:00.304458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 549/1082] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:11:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:11:00.312449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b4f85-628f-4ec4-8e29-24f43bf5df33 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:00.348055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988120d9-007c-425b-a671-65df4b6ed3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:00.349190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988120d9-007c-425b-a671-65df4b6ed3a5 None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:00.349685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988120d9-007c-425b-a671-65df4b6ed3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:00.350162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988120d9-007c-425b-a671-65df4b6ed3a5 None admin] PATH_INFO: `/v3/projects/7795a64328534f71b54602bddd5c0c37/users/66dde655d0e74efab8029d3cd47e0129/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:00.353666 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988120d9-007c-425b-a671-65df4b6ed3a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=7795a64328534f71b54602bddd5c0c37, user_id=66dde655d0e74efab8029d3cd47e0129, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:00.356247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988120d9-007c-425b-a671-65df4b6ed3a5 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:00.439120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b4f85-628f-4ec4-8e29-24f43bf5df33 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:00.448687 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 534/1083] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:11:00 2022] PUT /identity/v3/projects/7795a64328534f71b54602bddd5c0c37/users/66dde655d0e74efab8029d3cd47e0129/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:11:00.495630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aec46a-cead-41c3-a364-d2248329ecef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:00.496794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aec46a-cead-41c3-a364-d2248329ecef None admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:00.497265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aec46a-cead-41c3-a364-d2248329ecef None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:00.497814 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aec46a-cead-41c3-a364-d2248329ecef None admin] PATH_INFO: `/v3/projects/4574cf3ff270485faf60594019f7e05c/users/97c363046db444cbb3d4346066f9ad42/roles/e88e2f39fcb94166851f6db7b9f9dfcb` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:00.501689 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aec46a-cead-41c3-a364-d2248329ecef None admin] RBAC: Authorizing `identity:create_grant(project_id=4574cf3ff270485faf60594019f7e05c, user_id=97c363046db444cbb3d4346066f9ad42, role_id=e88e2f39fcb94166851f6db7b9f9dfcb)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:00.504578 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aec46a-cead-41c3-a364-d2248329ecef None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:00.604059 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 550/1084] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:11:00 2022] PUT /identity/v3/projects/4574cf3ff270485faf60594019f7e05c/users/97c363046db444cbb3d4346066f9ad42/roles/e88e2f39fcb94166851f6db7b9f9dfcb => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:11:02.880237 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c78483-16a9-485c-a876-fb1b7d0dc2ce None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:02.880860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c78483-16a9-485c-a876-fb1b7d0dc2ce None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:02.880860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c78483-16a9-485c-a876-fb1b7d0dc2ce None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:02.882411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 535/1085] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:11:02.888586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653d720-f811-40bb-942c-d68adb6fa533 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:02.889180 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653d720-f811-40bb-942c-d68adb6fa533 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:02.889312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653d720-f811-40bb-942c-d68adb6fa533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:02.912306 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b653d720-f811-40bb-942c-d68adb6fa533 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:02.913652 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b653d720-f811-40bb-942c-d68adb6fa533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:02.936761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:03.002035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 551/1086] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:11:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:11:03.298854 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c78483-16a9-485c-a876-fb1b7d0dc2ce None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:03.406993 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:03.467821 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c78483-16a9-485c-a876-fb1b7d0dc2ce None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:03.483593 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1da3be-9002-4e5e-939f-7efba8ad3e7d None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:03.486297 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1da3be-9002-4e5e-939f-7efba8ad3e7d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:03.486297 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1da3be-9002-4e5e-939f-7efba8ad3e7d None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:03.486297 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 552/1087] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:11:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:11:03.492341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdb1a2-e142-4f31-bc3f-f92abcf0cfb6 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:03.492621 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdb1a2-e142-4f31-bc3f-f92abcf0cfb6 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:03.492901 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdb1a2-e142-4f31-bc3f-f92abcf0cfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:03.515519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-99cdb1a2-e142-4f31-bc3f-f92abcf0cfb6 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:03.516756 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-99cdb1a2-e142-4f31-bc3f-f92abcf0cfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:03.522256 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef612c89-50c2-43aa-9848-e8103e982d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:03.522999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef612c89-50c2-43aa-9848-e8103e982d9e None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:03.523348 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef612c89-50c2-43aa-9848-e8103e982d9e None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:03.523680 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef612c89-50c2-43aa-9848-e8103e982d9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:03.541796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef612c89-50c2-43aa-9848-e8103e982d9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:11:03.567266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-ef612c89-50c2-43aa-9848-e8103e982d9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:11:03.567266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:11:03.567266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:11:03.567266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:11:03.567266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:11:03.567266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:11:03.567266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:11:03.567266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:11:03.567266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:11:03.567266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:11:03.567266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:11:03.567266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:11:03.567266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:11:03.567266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:11:03.567266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:11:03.567266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:11:03.570229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:11:03.570229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:11:03.570229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:11:03.570229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:11:03.570229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:11:03.570229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:11:03.570229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:11:03.570229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:11:03.570229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:11:03.570229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:11:03.570229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:11:03.570229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:11:03.570229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:11:03.570229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:11:03.570229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:11:03.570229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:11:03.572315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:11:03.572315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:11:03.572315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:11:03.572315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:11:03.572315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:11:03.572315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:11:03.572315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:11:03.572315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:11:03.572315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:11:03.572315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:11:03.572315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:11:03.572315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:11:03.572315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 536/1088] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:11:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:11:03.572315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef612c89-50c2-43aa-9848-e8103e982d9e None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:03.656135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 553/1089] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:11:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:11:03.719325 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef612c89-50c2-43aa-9848-e8103e982d9e None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:03.752564 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:03.783041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc9aa3-4b4f-4811-a14b-e5e2c70a5a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:03.784032 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc9aa3-4b4f-4811-a14b-e5e2c70a5a55 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:03.784585 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc9aa3-4b4f-4811-a14b-e5e2c70a5a55 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:03.785067 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc9aa3-4b4f-4811-a14b-e5e2c70a5a55 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:03.786278 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc9aa3-4b4f-4811-a14b-e5e2c70a5a55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:03.786816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc9aa3-4b4f-4811-a14b-e5e2c70a5a55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:03.789570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc9aa3-4b4f-4811-a14b-e5e2c70a5a55 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:03.805289 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 537/1090] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:11:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:03.813518 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cdb1a2-e142-4f31-bc3f-f92abcf0cfb6 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:03.961104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:03.986223 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cdb1a2-e142-4f31-bc3f-f92abcf0cfb6 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:04.032126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468effd-8d3b-4b5f-8162-642100c3195c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:04.032890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468effd-8d3b-4b5f-8162-642100c3195c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:04.033228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468effd-8d3b-4b5f-8162-642100c3195c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:04.033660 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468effd-8d3b-4b5f-8162-642100c3195c None admin] PATH_INFO: `/v3/users/66dde655d0e74efab8029d3cd47e0129` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:04.035534 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468effd-8d3b-4b5f-8162-642100c3195c None admin] RBAC: Authorizing `identity:get_user(user_id=66dde655d0e74efab8029d3cd47e0129)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:04.037092 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468effd-8d3b-4b5f-8162-642100c3195c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:04.038961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 554/1091] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:11:03 2022] GET /identity/v3/users/66dde655d0e74efab8029d3cd47e0129 => generated 285 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:04.045065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc9aa3-4b4f-4811-a14b-e5e2c70a5a55 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:04.046830 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468effd-8d3b-4b5f-8162-642100c3195c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:04.195360 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc9aa3-4b4f-4811-a14b-e5e2c70a5a55 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:04.204663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468effd-8d3b-4b5f-8162-642100c3195c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:04.241712 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df386a8c-862d-4bdf-a31a-d726c4da09d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:04.242446 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df386a8c-862d-4bdf-a31a-d726c4da09d1 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:04.242816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df386a8c-862d-4bdf-a31a-d726c4da09d1 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:04.243248 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df386a8c-862d-4bdf-a31a-d726c4da09d1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:04.250478 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc61da9-f926-42a3-8f06-dd1f0b03acc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:04.251322 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc61da9-f926-42a3-8f06-dd1f0b03acc3 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:04.251661 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc61da9-f926-42a3-8f06-dd1f0b03acc3 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:04.251935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc61da9-f926-42a3-8f06-dd1f0b03acc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:04.258392 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df386a8c-862d-4bdf-a31a-d726c4da09d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:11:04.264299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-df386a8c-862d-4bdf-a31a-d726c4da09d1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:11:04.264299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:11:04.264299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:11:04.264299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:11:04.264299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:11:04.264299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:11:04.264299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:11:04.264299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:11:04.264299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:11:04.264299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:11:04.264299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:11:04.264299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:11:04.264299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:11:04.264299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:11:04.264299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:11:04.264299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:11:04.266208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:11:04.266208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:11:04.266208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:11:04.266208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:11:04.266208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:11:04.266208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:11:04.266208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:11:04.266208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:11:04.266208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:11:04.266208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:11:04.266208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:11:04.266208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:11:04.266208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:11:04.266208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:11:04.266208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:11:04.266208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:11:04.268223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:11:04.268223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:11:04.268223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:11:04.268223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:11:04.268223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:11:04.268223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:11:04.268223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:11:04.268223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:11:04.268223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:11:04.268223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:11:04.268223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 13:11:04.268223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:11:04.268223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 538/1092] 10.222.0.62 () {56 vars in 1120 bytes} [Wed Sep 7 13:11:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:11:04.272520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df386a8c-862d-4bdf-a31a-d726c4da09d1 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:04.333989 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:04.347285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:04.423178 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc61da9-f926-42a3-8f06-dd1f0b03acc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:04.424669 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc61da9-f926-42a3-8f06-dd1f0b03acc3 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:04.428696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df386a8c-862d-4bdf-a31a-d726c4da09d1 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:04.471280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6deadf-91cb-4e50-b843-9f182ef97fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:04.471988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6deadf-91cb-4e50-b843-9f182ef97fe5 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:04.472272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6deadf-91cb-4e50-b843-9f182ef97fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:04.472543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6deadf-91cb-4e50-b843-9f182ef97fe5 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:04.473447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6deadf-91cb-4e50-b843-9f182ef97fe5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:04.473898 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6deadf-91cb-4e50-b843-9f182ef97fe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:04.478312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6deadf-91cb-4e50-b843-9f182ef97fe5 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:04.486045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 539/1093] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Sep 7 13:11:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:11:04.493045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6deadf-91cb-4e50-b843-9f182ef97fe5 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:04.590027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 555/1094] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:11:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 13:11:04.597300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc61da9-f926-42a3-8f06-dd1f0b03acc3 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:04.656321 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:04.657877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6deadf-91cb-4e50-b843-9f182ef97fe5 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:04.660827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:04.714409 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ceb66-d2f2-4c84-8607-de0d8758c579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:04.715356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ceb66-d2f2-4c84-8607-de0d8758c579 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:04.715701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ceb66-d2f2-4c84-8607-de0d8758c579 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:04.716034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ceb66-d2f2-4c84-8607-de0d8758c579 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:04.729290 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ceb66-d2f2-4c84-8607-de0d8758c579 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:04.730808 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ceb66-d2f2-4c84-8607-de0d8758c579 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:04.736266 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc61da9-f926-42a3-8f06-dd1f0b03acc3 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:04.749267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-034ceb66-d2f2-4c84-8607-de0d8758c579 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:11:04.749267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:11:04.749267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:11:04.749267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:11:04.749267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:11:04.749267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:11:04.749267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:11:04.749267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:11:04.749267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:11:04.749267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:11:04.749267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:11:04.749267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:11:04.749267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:11:04.749267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:11:04.749267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:11:04.749267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:11:04.751320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:11:04.751320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:11:04.751320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:11:04.751320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:11:04.751320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:11:04.751320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:11:04.751320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:11:04.751320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:11:04.751320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:11:04.751320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:11:04.751320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:11:04.751320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:11:04.751320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:11:04.751320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:11:04.751320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:11:04.751320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:11:04.751320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:11:04.753117 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:11:04.753117 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:11:04.753117 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:11:04.753117 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 540/1095] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:11:04.756837 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ceb66-d2f2-4c84-8607-de0d8758c579 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:04.786471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9200a72-e733-4386-8cdc-cf0cd522a741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:04.787161 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9200a72-e733-4386-8cdc-cf0cd522a741 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:04.787490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9200a72-e733-4386-8cdc-cf0cd522a741 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:04.787758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9200a72-e733-4386-8cdc-cf0cd522a741 None admin] PATH_INFO: `/v3/projects/7795a64328534f71b54602bddd5c0c37` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:04.788983 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9200a72-e733-4386-8cdc-cf0cd522a741 None admin] RBAC: Authorizing `identity:get_project(project_id=7795a64328534f71b54602bddd5c0c37)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:04.790463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9200a72-e733-4386-8cdc-cf0cd522a741 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:04.792405 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 556/1096] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:11:04 2022] GET /identity/v3/projects/7795a64328534f71b54602bddd5c0c37 => generated 311 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:11:04.800092 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9200a72-e733-4386-8cdc-cf0cd522a741 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:04.915381 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ceb66-d2f2-4c84-8607-de0d8758c579 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:04.956197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9200a72-e733-4386-8cdc-cf0cd522a741 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:04.963341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b82b2c-8865-4872-a491-c005003e2e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:04.964074 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b82b2c-8865-4872-a491-c005003e2e86 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:04.964473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b82b2c-8865-4872-a491-c005003e2e86 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:04.964858 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b82b2c-8865-4872-a491-c005003e2e86 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:04.965753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b82b2c-8865-4872-a491-c005003e2e86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:04.966062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b82b2c-8865-4872-a491-c005003e2e86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:04.967963 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b82b2c-8865-4872-a491-c005003e2e86 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:04.977386 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:04.989377 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 541/1097] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:04.997558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b77d6-42a7-43e6-a412-340329a574e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:04.998696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b77d6-42a7-43e6-a412-340329a574e5 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:04.999005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b77d6-42a7-43e6-a412-340329a574e5 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:04.999353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b82b2c-8865-4872-a491-c005003e2e86 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:04.999520 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b77d6-42a7-43e6-a412-340329a574e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:05.000475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b77d6-42a7-43e6-a412-340329a574e5 None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=7795a64328534f71b54602bddd5c0c37, user.id=66dde655d0e74efab8029d3cd47e0129) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:05.000795 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b77d6-42a7-43e6-a412-340329a574e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:05.002475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b77d6-42a7-43e6-a412-340329a574e5 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:05.009741 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 557/1098] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:11:04 2022] GET /identity/v3/role_assignments?user.id=66dde655d0e74efab8029d3cd47e0129&scope.project.id=7795a64328534f71b54602bddd5c0c37&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:11:05.031221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:05.164275 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b82b2c-8865-4872-a491-c005003e2e86 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:05.216723 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633be384-63fb-421a-a8ce-6a14f7cf5790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:05.217782 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633be384-63fb-421a-a8ce-6a14f7cf5790 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:05.218074 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633be384-63fb-421a-a8ce-6a14f7cf5790 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:05.218354 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633be384-63fb-421a-a8ce-6a14f7cf5790 None admin] PATH_INFO: `/v3/users/nova` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:05.229218 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633be384-63fb-421a-a8ce-6a14f7cf5790 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:05.231066 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633be384-63fb-421a-a8ce-6a14f7cf5790 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:05.255877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-633be384-63fb-421a-a8ce-6a14f7cf5790 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:11:05.255877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:11:05.255877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:11:05.255877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:11:05.255877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:11:05.255877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:11:05.255877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:11:05.255877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:11:05.255877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:11:05.255877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:11:05.255877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:11:05.255877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:11:05.255877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:11:05.255877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:11:05.255877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:11:05.255877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:11:05.257422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:11:05.259955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:11:05.259955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:11:05.259955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:11:05.259955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:11:05.259955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:11:05.259955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:11:05.259955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:11:05.259955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:11:05.259955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 13:11:05.259955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:11:05.259955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 542/1099] 10.222.0.62 () {56 vars in 1093 bytes} [Wed Sep 7 13:11:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:11:05.265838 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b77d6-42a7-43e6-a412-340329a574e5 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:05.438994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b77d6-42a7-43e6-a412-340329a574e5 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:05.508699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473024a-5908-486b-bf91-32b4ffe9e41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:05.509330 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473024a-5908-486b-bf91-32b4ffe9e41d None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:05.509851 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473024a-5908-486b-bf91-32b4ffe9e41d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:05.510288 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473024a-5908-486b-bf91-32b4ffe9e41d None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:05.511602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473024a-5908-486b-bf91-32b4ffe9e41d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:05.512153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473024a-5908-486b-bf91-32b4ffe9e41d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:05.515581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473024a-5908-486b-bf91-32b4ffe9e41d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:05.548948 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 558/1100] 10.222.0.62 () {56 vars in 1133 bytes} [Wed Sep 7 13:11:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:05.557584 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633be384-63fb-421a-a8ce-6a14f7cf5790 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:05.688435 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:05.720718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633be384-63fb-421a-a8ce-6a14f7cf5790 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:05.769989 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0ebc9-26fb-441c-b6b6-1acf5ef1854b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:05.770493 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0ebc9-26fb-441c-b6b6-1acf5ef1854b None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:05.770773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0ebc9-26fb-441c-b6b6-1acf5ef1854b None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:05.771106 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0ebc9-26fb-441c-b6b6-1acf5ef1854b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:05.859395 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd0ebc9-26fb-441c-b6b6-1acf5ef1854b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:05.861016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd0ebc9-26fb-441c-b6b6-1acf5ef1854b None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:05.952356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:05.984981 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 543/1101] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:11:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:11:05.991110 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473024a-5908-486b-bf91-32b4ffe9e41d None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:06.043544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:06.123575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473024a-5908-486b-bf91-32b4ffe9e41d None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:06.167347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9d16d-da44-44d5-a1c6-bd60d705c04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:06.168322 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9d16d-da44-44d5-a1c6-bd60d705c04c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:06.168827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9d16d-da44-44d5-a1c6-bd60d705c04c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:06.169729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9d16d-da44-44d5-a1c6-bd60d705c04c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:06.189686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9d16d-da44-44d5-a1c6-bd60d705c04c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:06.192402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9d16d-da44-44d5-a1c6-bd60d705c04c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:06.220597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-3fa9d16d-da44-44d5-a1c6-bd60d705c04c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:11:06.220597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:11:06.220597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:11:06.220597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:11:06.220597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:11:06.220597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:11:06.220597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:11:06.220597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:11:06.220597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:11:06.220597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:11:06.220597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:11:06.220597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:11:06.220597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:11:06.220597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:11:06.220597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:11:06.220597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:11:06.221936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:11:06.221936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:11:06.221936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:11:06.221936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:11:06.221936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:11:06.221936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:11:06.221936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:11:06.221936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:11:06.221936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:11:06.221936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:11:06.221936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:11:06.221936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:11:06.221936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:11:06.221936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:11:06.221936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:11:06.221936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:11:06.221936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:11:06.223897 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:11:06.223897 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:11:06.223897 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:11:06.223897 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 559/1102] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:11:06.229498 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0ebc9-26fb-441c-b6b6-1acf5ef1854b None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:06.361819 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0ebc9-26fb-441c-b6b6-1acf5ef1854b None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:06.407701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b42ec1-b433-4d30-a02d-285e02309ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:06.408470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b42ec1-b433-4d30-a02d-285e02309ab1 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:06.408843 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b42ec1-b433-4d30-a02d-285e02309ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:06.409210 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b42ec1-b433-4d30-a02d-285e02309ab1 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:06.410008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b42ec1-b433-4d30-a02d-285e02309ab1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:06.410403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b42ec1-b433-4d30-a02d-285e02309ab1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:06.411989 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b42ec1-b433-4d30-a02d-285e02309ab1 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:06.426852 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 544/1103] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:06.432703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9d16d-da44-44d5-a1c6-bd60d705c04c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:06.572102 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:06.582802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9d16d-da44-44d5-a1c6-bd60d705c04c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:06.626099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5450f0c-fcb1-4ffc-8cfb-a0183c68e7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:06.626645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5450f0c-fcb1-4ffc-8cfb-a0183c68e7b0 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:06.626919 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5450f0c-fcb1-4ffc-8cfb-a0183c68e7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:06.627224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5450f0c-fcb1-4ffc-8cfb-a0183c68e7b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:06.636054 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5450f0c-fcb1-4ffc-8cfb-a0183c68e7b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:06.637677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5450f0c-fcb1-4ffc-8cfb-a0183c68e7b0 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:06.652398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-f5450f0c-fcb1-4ffc-8cfb-a0183c68e7b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:11:06.652398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:11:06.652398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:11:06.652398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:11:06.652398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:11:06.652398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:11:06.652398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:11:06.652398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:11:06.652398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:11:06.652398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:11:06.652398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:11:06.652398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:11:06.652398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:11:06.652398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:11:06.652398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:11:06.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:11:06.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:11:06.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:11:06.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:11:06.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:11:06.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:11:06.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:11:06.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:11:06.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:11:06.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:11:06.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:11:06.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:11:06.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:11:06.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:11:06.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:11:06.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:11:06.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:11:06.656738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:11:06.656738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:11:06.656738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:11:06.656738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:11:06.656738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:11:06.656738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:11:06.656738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:11:06.656738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:11:06.656738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 560/1104] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:11:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:11:06.662768 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b42ec1-b433-4d30-a02d-285e02309ab1 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:06.711337 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:06.833489 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b42ec1-b433-4d30-a02d-285e02309ab1 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:06.896585 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe4f0c-f0e7-4dd4-80eb-9f33d6d03d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:06.897542 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe4f0c-f0e7-4dd4-80eb-9f33d6d03d21 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:06.898035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe4f0c-f0e7-4dd4-80eb-9f33d6d03d21 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:06.898533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe4f0c-f0e7-4dd4-80eb-9f33d6d03d21 None admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:06.899893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe4f0c-f0e7-4dd4-80eb-9f33d6d03d21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:06.900445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe4f0c-f0e7-4dd4-80eb-9f33d6d03d21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:06.903291 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe4f0c-f0e7-4dd4-80eb-9f33d6d03d21 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:06.927552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 545/1105] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:11:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:11:06.934537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5450f0c-fcb1-4ffc-8cfb-a0183c68e7b0 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:07.054622 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5450f0c-fcb1-4ffc-8cfb-a0183c68e7b0 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:07.093447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c7216-296b-424e-ab0b-f7d55c50665d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:07.094067 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c7216-296b-424e-ab0b-f7d55c50665d None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:07.094343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c7216-296b-424e-ab0b-f7d55c50665d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:07.094604 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c7216-296b-424e-ab0b-f7d55c50665d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:07.095388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c7216-296b-424e-ab0b-f7d55c50665d None admin] RBAC: Adding query filter params (role.id=e88e2f39fcb94166851f6db7b9f9dfcb, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=97c363046db444cbb3d4346066f9ad42) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:07.095670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c7216-296b-424e-ab0b-f7d55c50665d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:07.097123 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c7216-296b-424e-ab0b-f7d55c50665d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:07.103811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 561/1106] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:11:06 2022] GET /identity/v3/role_assignments?user.id=97c363046db444cbb3d4346066f9ad42&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=e88e2f39fcb94166851f6db7b9f9dfcb => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:07.678341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24907abd-2d73-482f-8d2d-d2fa012aa807 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:07.679155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24907abd-2d73-482f-8d2d-d2fa012aa807 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:07.679155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24907abd-2d73-482f-8d2d-d2fa012aa807 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:07.680497 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 546/1107] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:11:07.687998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b4295-96f4-4f0b-a561-25dc4fa9f32c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:07.688284 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b4295-96f4-4f0b-a561-25dc4fa9f32c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:07.688710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b4295-96f4-4f0b-a561-25dc4fa9f32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:07.711195 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-897b4295-96f4-4f0b-a561-25dc4fa9f32c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:07.712435 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-897b4295-96f4-4f0b-a561-25dc4fa9f32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:07.726866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:07.799236 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 562/1108] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:11:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:11:08.121028 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24907abd-2d73-482f-8d2d-d2fa012aa807 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:08.257741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:08.268436 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24907abd-2d73-482f-8d2d-d2fa012aa807 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:08.320917 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742ce60-3898-43cd-913a-2a52adcd9768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:08.321722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742ce60-3898-43cd-913a-2a52adcd9768 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:08.322109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742ce60-3898-43cd-913a-2a52adcd9768 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:08.322604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742ce60-3898-43cd-913a-2a52adcd9768 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:08.324448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742ce60-3898-43cd-913a-2a52adcd9768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:08.327456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742ce60-3898-43cd-913a-2a52adcd9768 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:08.330566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 547/1109] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:11:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:11:08.336383 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b4295-96f4-4f0b-a561-25dc4fa9f32c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:08.460764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b4295-96f4-4f0b-a561-25dc4fa9f32c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:08.498581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45293d-694c-4812-8d6f-62bb5b88bde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:08.499360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45293d-694c-4812-8d6f-62bb5b88bde7 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:08.499729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45293d-694c-4812-8d6f-62bb5b88bde7 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:08.500103 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45293d-694c-4812-8d6f-62bb5b88bde7 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:08.500877 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45293d-694c-4812-8d6f-62bb5b88bde7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:08.502399 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db45293d-694c-4812-8d6f-62bb5b88bde7 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:08.671254 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 563/1110] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:11:08 2022] POST /identity/v3/users => generated 286 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:11:11.188627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683f2fd-efd2-4083-8c40-6bd11d5ffa7f None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:11.189293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683f2fd-efd2-4083-8c40-6bd11d5ffa7f None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:11.189610 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683f2fd-efd2-4083-8c40-6bd11d5ffa7f None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:11.192193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 548/1111] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:11:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:11:11.199919 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d57a6-25bb-4d70-92f2-a8c9f97d30d8 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:11.201098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d57a6-25bb-4d70-92f2-a8c9f97d30d8 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:11.201585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d57a6-25bb-4d70-92f2-a8c9f97d30d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:11.212905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:11.236564 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-ff8d57a6-25bb-4d70-92f2-a8c9f97d30d8 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:11.238666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ff8d57a6-25bb-4d70-92f2-a8c9f97d30d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:11.377430 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 564/1112] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:11:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:11:11.918594 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683f2fd-efd2-4083-8c40-6bd11d5ffa7f None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:12.100609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683f2fd-efd2-4083-8c40-6bd11d5ffa7f None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:12.140939 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa664a0-a75c-429e-b2d7-c168cd731981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:12.141584 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa664a0-a75c-429e-b2d7-c168cd731981 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:12.141860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa664a0-a75c-429e-b2d7-c168cd731981 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:12.142128 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa664a0-a75c-429e-b2d7-c168cd731981 None admin] PATH_INFO: `/v3/roles/fc0a7877dd0f4903a2a6fcc01a829c6a` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:12.143645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa664a0-a75c-429e-b2d7-c168cd731981 None admin] RBAC: Authorizing `identity:get_role(role_id=fc0a7877dd0f4903a2a6fcc01a829c6a)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:12.145037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa664a0-a75c-429e-b2d7-c168cd731981 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:12.146407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 549/1113] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:11:11 2022] GET /identity/v3/roles/fc0a7877dd0f4903a2a6fcc01a829c6a => generated 223 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:12.152691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d57a6-25bb-4d70-92f2-a8c9f97d30d8 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:12.282481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d57a6-25bb-4d70-92f2-a8c9f97d30d8 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:12.318327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:12.323791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070a3be-cf02-46f7-ba2e-5b30130d4e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:12.324559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070a3be-cf02-46f7-ba2e-5b30130d4e48 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:12.342740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070a3be-cf02-46f7-ba2e-5b30130d4e48 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:12.342740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070a3be-cf02-46f7-ba2e-5b30130d4e48 None admin] PATH_INFO: `/v3/users/88512ab5be834e0a92d1262c7a94cb0f` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:12.343786 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070a3be-cf02-46f7-ba2e-5b30130d4e48 None admin] RBAC: Authorizing `identity:get_user(user_id=88512ab5be834e0a92d1262c7a94cb0f)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:12.345258 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070a3be-cf02-46f7-ba2e-5b30130d4e48 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:12.347062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 565/1114] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:11:12 2022] GET /identity/v3/users/88512ab5be834e0a92d1262c7a94cb0f => generated 286 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:11:12.353304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa664a0-a75c-429e-b2d7-c168cd731981 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:12.393011 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:12.481328 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa664a0-a75c-429e-b2d7-c168cd731981 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:12.522750 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e765dd6-d482-4e4e-9f2c-b24ff0f6cf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:12.525719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e765dd6-d482-4e4e-9f2c-b24ff0f6cf23 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:12.525719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e765dd6-d482-4e4e-9f2c-b24ff0f6cf23 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:12.525719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e765dd6-d482-4e4e-9f2c-b24ff0f6cf23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:12.620343 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e765dd6-d482-4e4e-9f2c-b24ff0f6cf23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:12.622929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e765dd6-d482-4e4e-9f2c-b24ff0f6cf23 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:12.667454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:12.758605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 550/1115] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:11:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:11:12.766008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070a3be-cf02-46f7-ba2e-5b30130d4e48 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:12.884065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070a3be-cf02-46f7-ba2e-5b30130d4e48 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:12.926056 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b0767-8f0b-4c7c-97af-a817346a7687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:12.926714 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b0767-8f0b-4c7c-97af-a817346a7687 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:12.927021 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b0767-8f0b-4c7c-97af-a817346a7687 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:12.927291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b0767-8f0b-4c7c-97af-a817346a7687 None admin] PATH_INFO: `/v3/projects/7795a64328534f71b54602bddd5c0c37` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:12.928485 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b0767-8f0b-4c7c-97af-a817346a7687 None admin] RBAC: Authorizing `identity:get_project(project_id=7795a64328534f71b54602bddd5c0c37)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:12.929948 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b0767-8f0b-4c7c-97af-a817346a7687 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:12.931751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 566/1116] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:11:12 2022] GET /identity/v3/projects/7795a64328534f71b54602bddd5c0c37 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:12.939769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e765dd6-d482-4e4e-9f2c-b24ff0f6cf23 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:13.121126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e765dd6-d482-4e4e-9f2c-b24ff0f6cf23 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:13.179708 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:13.190551 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ebcd9-35c8-4d60-a9c8-bff4e4dd3ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:13.195765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ebcd9-35c8-4d60-a9c8-bff4e4dd3ecd None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:13.195765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ebcd9-35c8-4d60-a9c8-bff4e4dd3ecd None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:13.195765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ebcd9-35c8-4d60-a9c8-bff4e4dd3ecd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:13.195765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ebcd9-35c8-4d60-a9c8-bff4e4dd3ecd None admin] RBAC: Adding query filter params (role.id=fc0a7877dd0f4903a2a6fcc01a829c6a, scope.project.id=7795a64328534f71b54602bddd5c0c37, user.id=88512ab5be834e0a92d1262c7a94cb0f) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:13.195765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ebcd9-35c8-4d60-a9c8-bff4e4dd3ecd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:13.197338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ebcd9-35c8-4d60-a9c8-bff4e4dd3ecd None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:13.208923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 551/1117] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:11:12 2022] GET /identity/v3/role_assignments?user.id=88512ab5be834e0a92d1262c7a94cb0f&scope.project.id=7795a64328534f71b54602bddd5c0c37&role.id=fc0a7877dd0f4903a2a6fcc01a829c6a => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:15.731301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486d902-1289-4cda-94ac-4e8ae4ed0915 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:15.731923 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486d902-1289-4cda-94ac-4e8ae4ed0915 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:15.732270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486d902-1289-4cda-94ac-4e8ae4ed0915 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:15.734722 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 567/1118] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:11:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:11:15.741290 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff469a7a-63f6-45ed-a6ed-f7bfc0035093 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:15.741827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff469a7a-63f6-45ed-a6ed-f7bfc0035093 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:15.741959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff469a7a-63f6-45ed-a6ed-f7bfc0035093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:15.769117 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-ff469a7a-63f6-45ed-a6ed-f7bfc0035093 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:15.770358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ff469a7a-63f6-45ed-a6ed-f7bfc0035093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:15.848891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 552/1119] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:11:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:11:16.128060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486d902-1289-4cda-94ac-4e8ae4ed0915 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:16.244147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:16.331832 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486d902-1289-4cda-94ac-4e8ae4ed0915 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:16.397564 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195d2f7-d619-46f1-a552-457b84695503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:16.398561 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195d2f7-d619-46f1-a552-457b84695503 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:16.399046 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195d2f7-d619-46f1-a552-457b84695503 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:16.399542 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195d2f7-d619-46f1-a552-457b84695503 None admin] PATH_INFO: `/v3/roles/fc0a7877dd0f4903a2a6fcc01a829c6a` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:16.402341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195d2f7-d619-46f1-a552-457b84695503 None admin] RBAC: Authorizing `identity:get_role(role_id=fc0a7877dd0f4903a2a6fcc01a829c6a)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:16.405766 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195d2f7-d619-46f1-a552-457b84695503 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:16.408343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 568/1120] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:11:16 2022] GET /identity/v3/roles/fc0a7877dd0f4903a2a6fcc01a829c6a => generated 223 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:16.414594 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff469a7a-63f6-45ed-a6ed-f7bfc0035093 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:16.611546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff469a7a-63f6-45ed-a6ed-f7bfc0035093 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:16.657988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:16.676684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0b2cb-fd36-452c-b4d6-0f2b2c25e342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:16.677757 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0b2cb-fd36-452c-b4d6-0f2b2c25e342 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:16.678241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0b2cb-fd36-452c-b4d6-0f2b2c25e342 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:16.678714 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0b2cb-fd36-452c-b4d6-0f2b2c25e342 None admin] PATH_INFO: `/v3/users/88512ab5be834e0a92d1262c7a94cb0f` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:16.680793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0b2cb-fd36-452c-b4d6-0f2b2c25e342 None admin] RBAC: Authorizing `identity:get_user(user_id=88512ab5be834e0a92d1262c7a94cb0f)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:16.683588 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0b2cb-fd36-452c-b4d6-0f2b2c25e342 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:16.687890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 553/1121] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:11:16 2022] GET /identity/v3/users/88512ab5be834e0a92d1262c7a94cb0f => generated 286 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:11:16.693788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195d2f7-d619-46f1-a552-457b84695503 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:16.866819 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195d2f7-d619-46f1-a552-457b84695503 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:16.897501 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:16.916457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8029365-4d6a-402e-977a-7871a06afe53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:16.916457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8029365-4d6a-402e-977a-7871a06afe53 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:16.916457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8029365-4d6a-402e-977a-7871a06afe53 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:16.916457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8029365-4d6a-402e-977a-7871a06afe53 None admin] PATH_INFO: `/v3/projects/7795a64328534f71b54602bddd5c0c37` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:16.917313 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8029365-4d6a-402e-977a-7871a06afe53 None admin] RBAC: Authorizing `identity:get_project(project_id=7795a64328534f71b54602bddd5c0c37)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:16.917313 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8029365-4d6a-402e-977a-7871a06afe53 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:16.917313 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 569/1122] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:11:16 2022] GET /identity/v3/projects/7795a64328534f71b54602bddd5c0c37 => generated 311 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:16.921039 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0b2cb-fd36-452c-b4d6-0f2b2c25e342 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:17.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0b2cb-fd36-452c-b4d6-0f2b2c25e342 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:17.136657 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82189f39-f523-4494-a9d3-3a8b75ecb6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:17.137368 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82189f39-f523-4494-a9d3-3a8b75ecb6ff None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:17.137681 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82189f39-f523-4494-a9d3-3a8b75ecb6ff None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:17.137979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82189f39-f523-4494-a9d3-3a8b75ecb6ff None admin] PATH_INFO: `/v3/projects/7795a64328534f71b54602bddd5c0c37/users/88512ab5be834e0a92d1262c7a94cb0f/roles/fc0a7877dd0f4903a2a6fcc01a829c6a` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:17.140670 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82189f39-f523-4494-a9d3-3a8b75ecb6ff None admin] RBAC: Authorizing `identity:create_grant(project_id=7795a64328534f71b54602bddd5c0c37, user_id=88512ab5be834e0a92d1262c7a94cb0f, role_id=fc0a7877dd0f4903a2a6fcc01a829c6a)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:17.142167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82189f39-f523-4494-a9d3-3a8b75ecb6ff None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:17.247974 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 554/1123] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:11:16 2022] PUT /identity/v3/projects/7795a64328534f71b54602bddd5c0c37/users/88512ab5be834e0a92d1262c7a94cb0f/roles/fc0a7877dd0f4903a2a6fcc01a829c6a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:11:20.430370 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9399d-479f-46b9-b8a7-085c65753a14 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:20.431380 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9399d-479f-46b9-b8a7-085c65753a14 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:20.431380 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9399d-479f-46b9-b8a7-085c65753a14 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:20.432508 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 570/1124] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:11:20.439256 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1e99b-ff3a-4190-9096-63b449594351 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:20.439889 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1e99b-ff3a-4190-9096-63b449594351 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:20.439889 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1e99b-ff3a-4190-9096-63b449594351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:20.460798 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-0cb1e99b-ff3a-4190-9096-63b449594351 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:20.462079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-0cb1e99b-ff3a-4190-9096-63b449594351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:20.514917 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:20.537035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 555/1125] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:11:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:11:20.810172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9399d-479f-46b9-b8a7-085c65753a14 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:20.951850 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9399d-479f-46b9-b8a7-085c65753a14 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:20.999083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa8f4d-e611-496b-93b6-73d3e97394c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:20.999751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa8f4d-e611-496b-93b6-73d3e97394c6 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:21.000037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa8f4d-e611-496b-93b6-73d3e97394c6 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:21.000293 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa8f4d-e611-496b-93b6-73d3e97394c6 None admin] PATH_INFO: `/v3/roles/fc0a7877dd0f4903a2a6fcc01a829c6a` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:21.001856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa8f4d-e611-496b-93b6-73d3e97394c6 None admin] RBAC: Authorizing `identity:get_role(role_id=fc0a7877dd0f4903a2a6fcc01a829c6a)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:21.003285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa8f4d-e611-496b-93b6-73d3e97394c6 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:21.004715 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 571/1126] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:11:20 2022] GET /identity/v3/roles/fc0a7877dd0f4903a2a6fcc01a829c6a => generated 223 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:21.014165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1e99b-ff3a-4190-9096-63b449594351 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:21.138295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1e99b-ff3a-4190-9096-63b449594351 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:21.198357 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4824c-5b2f-4bad-af62-d2c57a60565e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:21.199110 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4824c-5b2f-4bad-af62-d2c57a60565e None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:21.199546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4824c-5b2f-4bad-af62-d2c57a60565e None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:21.200610 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4824c-5b2f-4bad-af62-d2c57a60565e None admin] PATH_INFO: `/v3/users/88512ab5be834e0a92d1262c7a94cb0f` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:21.202679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4824c-5b2f-4bad-af62-d2c57a60565e None admin] RBAC: Authorizing `identity:get_user(user_id=88512ab5be834e0a92d1262c7a94cb0f)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:21.205638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4824c-5b2f-4bad-af62-d2c57a60565e None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:21.208695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 556/1127] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:11:21 2022] GET /identity/v3/users/88512ab5be834e0a92d1262c7a94cb0f => generated 286 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:21.216132 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa8f4d-e611-496b-93b6-73d3e97394c6 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:21.304076 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:21.349377 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa8f4d-e611-496b-93b6-73d3e97394c6 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:21.394390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984a3e-df58-4a10-9589-0132bdfdd500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:21.395078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984a3e-df58-4a10-9589-0132bdfdd500 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:21.395359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984a3e-df58-4a10-9589-0132bdfdd500 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:21.395648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984a3e-df58-4a10-9589-0132bdfdd500 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:21.513634 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7984a3e-df58-4a10-9589-0132bdfdd500 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:21.516227 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7984a3e-df58-4a10-9589-0132bdfdd500 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:21.642928 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:21.709058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 572/1128] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:11:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:11:21.715474 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4824c-5b2f-4bad-af62-d2c57a60565e None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:21.793211 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:21.868096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4824c-5b2f-4bad-af62-d2c57a60565e None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:21.908729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e2164-0908-4beb-82d5-0319fdf45ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:21.909376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e2164-0908-4beb-82d5-0319fdf45ac6 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:21.909657 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e2164-0908-4beb-82d5-0319fdf45ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:21.909937 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e2164-0908-4beb-82d5-0319fdf45ac6 None admin] PATH_INFO: `/v3/projects/7795a64328534f71b54602bddd5c0c37` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:21.911191 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e2164-0908-4beb-82d5-0319fdf45ac6 None admin] RBAC: Authorizing `identity:get_project(project_id=7795a64328534f71b54602bddd5c0c37)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:21.912740 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e2164-0908-4beb-82d5-0319fdf45ac6 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:21.914596 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 557/1129] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:11:21 2022] GET /identity/v3/projects/7795a64328534f71b54602bddd5c0c37 => generated 311 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:21.925136 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984a3e-df58-4a10-9589-0132bdfdd500 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:22.048668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984a3e-df58-4a10-9589-0132bdfdd500 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:22.099287 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef58a7-799b-46a4-8948-5e8965fd3d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:22.100039 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef58a7-799b-46a4-8948-5e8965fd3d0e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:22.100320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef58a7-799b-46a4-8948-5e8965fd3d0e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:22.100658 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef58a7-799b-46a4-8948-5e8965fd3d0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:22.101481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef58a7-799b-46a4-8948-5e8965fd3d0e None admin] RBAC: Adding query filter params (role.id=fc0a7877dd0f4903a2a6fcc01a829c6a, scope.project.id=7795a64328534f71b54602bddd5c0c37, user.id=88512ab5be834e0a92d1262c7a94cb0f) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:22.101837 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef58a7-799b-46a4-8948-5e8965fd3d0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:22.103499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef58a7-799b-46a4-8948-5e8965fd3d0e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:22.111497 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 573/1130] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:11:21 2022] GET /identity/v3/role_assignments?user.id=88512ab5be834e0a92d1262c7a94cb0f&scope.project.id=7795a64328534f71b54602bddd5c0c37&role.id=fc0a7877dd0f4903a2a6fcc01a829c6a => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:25.011847 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de03bd-e1bd-43f4-a94a-7dd8f13ae957 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:25.012446 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de03bd-e1bd-43f4-a94a-7dd8f13ae957 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:25.012446 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de03bd-e1bd-43f4-a94a-7dd8f13ae957 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:25.013822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 558/1131] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:11:25.020646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecce50-8df6-4e1e-bd46-dbdbfae3a6cc None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:25.021308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecce50-8df6-4e1e-bd46-dbdbfae3a6cc None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:25.021566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecce50-8df6-4e1e-bd46-dbdbfae3a6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:25.052848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-29ecce50-8df6-4e1e-bd46-dbdbfae3a6cc None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:25.054513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-29ecce50-8df6-4e1e-bd46-dbdbfae3a6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:25.088055 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:25.138115 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 574/1132] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:11:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:11:25.405949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de03bd-e1bd-43f4-a94a-7dd8f13ae957 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:25.558794 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de03bd-e1bd-43f4-a94a-7dd8f13ae957 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:25.589474 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:25.616547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7f338-aa27-4812-b97b-44038e7ca217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:25.617240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7f338-aa27-4812-b97b-44038e7ca217 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:25.617521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7f338-aa27-4812-b97b-44038e7ca217 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:25.617826 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7f338-aa27-4812-b97b-44038e7ca217 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:25.619182 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7f338-aa27-4812-b97b-44038e7ca217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:25.620687 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7f338-aa27-4812-b97b-44038e7ca217 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:25.622884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 559/1133] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:11:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:25.629086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ecce50-8df6-4e1e-bd46-dbdbfae3a6cc None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:25.752785 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ecce50-8df6-4e1e-bd46-dbdbfae3a6cc None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:25.793149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530920bb-7e10-4b05-acdb-8eb5fabc262f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:25.794471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530920bb-7e10-4b05-acdb-8eb5fabc262f None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:25.795238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530920bb-7e10-4b05-acdb-8eb5fabc262f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:25.795238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530920bb-7e10-4b05-acdb-8eb5fabc262f None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:25.795694 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530920bb-7e10-4b05-acdb-8eb5fabc262f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:25.797171 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530920bb-7e10-4b05-acdb-8eb5fabc262f None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:25.909160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 575/1134] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 7 13:11:25 2022] POST /identity/v3/projects => generated 311 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:11:28.555254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d843c-949b-45ea-be71-14c3c97bb05b None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:28.555827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d843c-949b-45ea-be71-14c3c97bb05b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:28.555827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d843c-949b-45ea-be71-14c3c97bb05b None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:28.557178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 560/1135] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:11:28.563590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced186ff-6559-456d-92c0-30a572c5446b None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:28.564345 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced186ff-6559-456d-92c0-30a572c5446b None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:28.564345 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced186ff-6559-456d-92c0-30a572c5446b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:28.586832 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-ced186ff-6559-456d-92c0-30a572c5446b None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:28.588233 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ced186ff-6559-456d-92c0-30a572c5446b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:28.590496 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:28.663637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 576/1136] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:11:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:11:28.944003 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d843c-949b-45ea-be71-14c3c97bb05b None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:29.080300 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d843c-949b-45ea-be71-14c3c97bb05b None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:29.124624 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc2a90-56ec-4bd5-a730-3e2376a65ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:29.125336 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc2a90-56ec-4bd5-a730-3e2376a65ce3 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:29.125663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc2a90-56ec-4bd5-a730-3e2376a65ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:29.125985 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc2a90-56ec-4bd5-a730-3e2376a65ce3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:29.127343 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc2a90-56ec-4bd5-a730-3e2376a65ce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:29.128849 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc2a90-56ec-4bd5-a730-3e2376a65ce3 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:29.130608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 561/1137] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:11:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:11:29.138419 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced186ff-6559-456d-92c0-30a572c5446b None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:29.264320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced186ff-6559-456d-92c0-30a572c5446b None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:29.315753 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c353814-1160-4933-bedc-0f8cdb94eb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:29.316381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c353814-1160-4933-bedc-0f8cdb94eb73 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:29.316656 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c353814-1160-4933-bedc-0f8cdb94eb73 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:29.316930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c353814-1160-4933-bedc-0f8cdb94eb73 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:29.317614 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c353814-1160-4933-bedc-0f8cdb94eb73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:29.319123 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c353814-1160-4933-bedc-0f8cdb94eb73 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:29.334754 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:29.521731 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 577/1138] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:11:29 2022] POST /identity/v3/users => generated 286 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:11:31.906706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da6907-198a-404c-b3e4-dc6e4e9d8106 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:31.907924 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da6907-198a-404c-b3e4-dc6e4e9d8106 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:31.907924 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da6907-198a-404c-b3e4-dc6e4e9d8106 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:31.909343 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 562/1139] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:11:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:11:31.916443 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64415553-07c4-482d-90f8-fde05d08ae3a None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:31.916981 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64415553-07c4-482d-90f8-fde05d08ae3a None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:31.917385 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64415553-07c4-482d-90f8-fde05d08ae3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:31.948613 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-64415553-07c4-482d-90f8-fde05d08ae3a None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:31.952126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-64415553-07c4-482d-90f8-fde05d08ae3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:32.077998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 578/1140] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:11:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:11:32.363652 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da6907-198a-404c-b3e4-dc6e4e9d8106 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:32.427466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:32.482742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da6907-198a-404c-b3e4-dc6e4e9d8106 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:32.521466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5181e9-f39f-4fc8-8d51-5a454fe9069d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:32.522095 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5181e9-f39f-4fc8-8d51-5a454fe9069d None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:32.522372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5181e9-f39f-4fc8-8d51-5a454fe9069d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:32.522639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5181e9-f39f-4fc8-8d51-5a454fe9069d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:32.535394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5181e9-f39f-4fc8-8d51-5a454fe9069d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:11:32.540229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-7f5181e9-f39f-4fc8-8d51-5a454fe9069d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:11:32.540229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:11:32.540229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:11:32.540229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:11:32.540229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:11:32.540229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:11:32.540229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:11:32.540229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:11:32.540229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:11:32.540229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:11:32.540229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:11:32.540229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:11:32.540229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:11:32.540229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:11:32.540229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:11:32.540229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:11:32.541511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:11:32.541511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:11:32.541511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:11:32.541511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:11:32.541511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:11:32.541511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:11:32.541511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:11:32.541511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:11:32.541511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:11:32.541511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:11:32.541511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:11:32.541511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:11:32.541511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:11:32.541511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:11:32.541511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:11:32.541511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:11:32.542373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:11:32.542373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:11:32.542373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:11:32.542373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:11:32.542373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:11:32.542373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:11:32.542373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:11:32.542373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:11:32.542373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:11:32.542373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:11:32.542373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:11:32.542373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:11:32.542373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 563/1141] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:11:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:11:32.547988 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64415553-07c4-482d-90f8-fde05d08ae3a None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:32.671250 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64415553-07c4-482d-90f8-fde05d08ae3a None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:32.705728 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:32.717976 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc14f3-bda3-4e9c-bece-82ceaeb6b4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:32.718598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc14f3-bda3-4e9c-bece-82ceaeb6b4f3 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:32.718875 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc14f3-bda3-4e9c-bece-82ceaeb6b4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:32.719175 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc14f3-bda3-4e9c-bece-82ceaeb6b4f3 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:32.719849 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc14f3-bda3-4e9c-bece-82ceaeb6b4f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:32.720153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc14f3-bda3-4e9c-bece-82ceaeb6b4f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:32.721519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc14f3-bda3-4e9c-bece-82ceaeb6b4f3 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:32.731292 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 579/1142] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:11:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:32.738340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5181e9-f39f-4fc8-8d51-5a454fe9069d None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:32.851874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5181e9-f39f-4fc8-8d51-5a454fe9069d None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:32.869680 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:32.894295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ec7fd-0e46-4de4-ac7f-3a8ecbbd9824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:32.894998 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ec7fd-0e46-4de4-ac7f-3a8ecbbd9824 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:32.895295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ec7fd-0e46-4de4-ac7f-3a8ecbbd9824 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:32.895558 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ec7fd-0e46-4de4-ac7f-3a8ecbbd9824 None admin] PATH_INFO: `/v3/users/3e5cad8a50a44f78b8ab4e109fe7e0a5` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:32.912777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ec7fd-0e46-4de4-ac7f-3a8ecbbd9824 None admin] RBAC: Authorizing `identity:get_user(user_id=3e5cad8a50a44f78b8ab4e109fe7e0a5)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:32.914305 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ec7fd-0e46-4de4-ac7f-3a8ecbbd9824 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:32.916148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 564/1143] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:11:32 2022] GET /identity/v3/users/3e5cad8a50a44f78b8ab4e109fe7e0a5 => generated 286 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:32.923183 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc14f3-bda3-4e9c-bece-82ceaeb6b4f3 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:33.046652 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc14f3-bda3-4e9c-bece-82ceaeb6b4f3 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:33.086806 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fdd5d-5adb-4c00-b85e-6b836f39f2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:33.087460 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fdd5d-5adb-4c00-b85e-6b836f39f2cd None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:33.087743 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fdd5d-5adb-4c00-b85e-6b836f39f2cd None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:33.088016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fdd5d-5adb-4c00-b85e-6b836f39f2cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:33.150374 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:33.165153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fdd5d-5adb-4c00-b85e-6b836f39f2cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:33.166597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fdd5d-5adb-4c00-b85e-6b836f39f2cd None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:33.273282 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 580/1144] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:11:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 13:11:33.280836 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ec7fd-0e46-4de4-ac7f-3a8ecbbd9824 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:33.406427 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ec7fd-0e46-4de4-ac7f-3a8ecbbd9824 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:33.452948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667874f5-2bab-4a62-9c99-fdf657bf2da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:33.453898 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667874f5-2bab-4a62-9c99-fdf657bf2da1 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:33.454178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667874f5-2bab-4a62-9c99-fdf657bf2da1 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:33.454451 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667874f5-2bab-4a62-9c99-fdf657bf2da1 None admin] PATH_INFO: `/v3/projects/352d9e66fb14400c8c46b34ae374a38b` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:33.455793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667874f5-2bab-4a62-9c99-fdf657bf2da1 None admin] RBAC: Authorizing `identity:get_project(project_id=352d9e66fb14400c8c46b34ae374a38b)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:33.457263 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667874f5-2bab-4a62-9c99-fdf657bf2da1 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:33.459054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 565/1145] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:11:33 2022] GET /identity/v3/projects/352d9e66fb14400c8c46b34ae374a38b => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:11:33.466160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fdd5d-5adb-4c00-b85e-6b836f39f2cd None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:33.566771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:33.588097 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fdd5d-5adb-4c00-b85e-6b836f39f2cd None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:33.634718 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737b3f7-c0be-4317-8aef-0fbdbccc46ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:33.635457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737b3f7-c0be-4317-8aef-0fbdbccc46ec None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:33.635931 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737b3f7-c0be-4317-8aef-0fbdbccc46ec None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:33.636367 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737b3f7-c0be-4317-8aef-0fbdbccc46ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:33.637284 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f737b3f7-c0be-4317-8aef-0fbdbccc46ec None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=352d9e66fb14400c8c46b34ae374a38b, user.id=3e5cad8a50a44f78b8ab4e109fe7e0a5) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:33.637600 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f737b3f7-c0be-4317-8aef-0fbdbccc46ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:33.639634 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f737b3f7-c0be-4317-8aef-0fbdbccc46ec None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:33.647088 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 581/1146] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:11:33 2022] GET /identity/v3/role_assignments?user.id=3e5cad8a50a44f78b8ab4e109fe7e0a5&scope.project.id=352d9e66fb14400c8c46b34ae374a38b&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:35.854501 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd1ac2-0bf0-4aac-b6d9-77ba2cb7c7bd None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:35.855179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd1ac2-0bf0-4aac-b6d9-77ba2cb7c7bd None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:35.855179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd1ac2-0bf0-4aac-b6d9-77ba2cb7c7bd None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:35.857287 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 566/1147] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:11:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:11:35.863975 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f709d3-ea26-4573-a752-b12093c559e0 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:35.864526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f709d3-ea26-4573-a752-b12093c559e0 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:35.864526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f709d3-ea26-4573-a752-b12093c559e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:35.885949 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-47f709d3-ea26-4573-a752-b12093c559e0 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:35.887949 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-47f709d3-ea26-4573-a752-b12093c559e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:35.965231 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 582/1148] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:11:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:11:36.237140 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd1ac2-0bf0-4aac-b6d9-77ba2cb7c7bd None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:36.284244 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:36.348706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd1ac2-0bf0-4aac-b6d9-77ba2cb7c7bd None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:36.384802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb4e20-3e7e-40b7-92d4-e0f442218d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:36.385423 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb4e20-3e7e-40b7-92d4-e0f442218d81 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:36.385697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb4e20-3e7e-40b7-92d4-e0f442218d81 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:36.385963 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb4e20-3e7e-40b7-92d4-e0f442218d81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:36.396199 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb4e20-3e7e-40b7-92d4-e0f442218d81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:11:36.401037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-84bb4e20-3e7e-40b7-92d4-e0f442218d81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:11:36.401037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:11:36.401037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:11:36.401037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:11:36.401037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:11:36.401037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:11:36.401037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:11:36.401037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:11:36.401037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:11:36.401037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:11:36.401037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:11:36.401037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:11:36.401037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:11:36.401037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:11:36.401037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:11:36.401037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:11:36.402142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:11:36.402142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:11:36.402142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:11:36.402142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:11:36.402142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:11:36.402142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:11:36.402142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:11:36.402142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:11:36.402142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:11:36.402142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:11:36.402142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:11:36.402142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:11:36.402142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:11:36.402142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:11:36.402142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:11:36.402142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:11:36.403146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:11:36.403146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:11:36.403146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:11:36.403146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:11:36.403146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:11:36.403146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:11:36.403146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:11:36.403146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:11:36.403146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:11:36.403146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:11:36.403146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:11:36.403146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:11:36.403146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 567/1149] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:11:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:11:36.408447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f709d3-ea26-4573-a752-b12093c559e0 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:36.489016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:36.508355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f709d3-ea26-4573-a752-b12093c559e0 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:36.544025 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c59130-c791-426a-89a0-e240d439035f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:36.544632 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c59130-c791-426a-89a0-e240d439035f None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:36.544918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c59130-c791-426a-89a0-e240d439035f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:36.545184 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c59130-c791-426a-89a0-e240d439035f None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:36.545866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c59130-c791-426a-89a0-e240d439035f None admin] RBAC: Adding query filter params (name=admin) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:36.546164 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c59130-c791-426a-89a0-e240d439035f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:36.547581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c59130-c791-426a-89a0-e240d439035f None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:36.556786 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 583/1150] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:11:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:36.564010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb4e20-3e7e-40b7-92d4-e0f442218d81 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:36.677746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:36.680829 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb4e20-3e7e-40b7-92d4-e0f442218d81 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:36.720739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f99129-4132-4082-8ed6-041e2fc9f303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:36.721396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f99129-4132-4082-8ed6-041e2fc9f303 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:36.721674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f99129-4132-4082-8ed6-041e2fc9f303 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:36.721947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f99129-4132-4082-8ed6-041e2fc9f303 None admin] PATH_INFO: `/v3/users/3e5cad8a50a44f78b8ab4e109fe7e0a5` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:36.723182 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f99129-4132-4082-8ed6-041e2fc9f303 None admin] RBAC: Authorizing `identity:get_user(user_id=3e5cad8a50a44f78b8ab4e109fe7e0a5)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:36.724635 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f99129-4132-4082-8ed6-041e2fc9f303 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:36.726273 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 568/1151] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:11:36 2022] GET /identity/v3/users/3e5cad8a50a44f78b8ab4e109fe7e0a5 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:36.732530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c59130-c791-426a-89a0-e240d439035f None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:36.850176 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c59130-c791-426a-89a0-e240d439035f None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:36.889992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4e5a6-a6ba-40fb-8d5a-0c5a74e76445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:36.890800 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4e5a6-a6ba-40fb-8d5a-0c5a74e76445 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:36.890944 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4e5a6-a6ba-40fb-8d5a-0c5a74e76445 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:36.891249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4e5a6-a6ba-40fb-8d5a-0c5a74e76445 None admin] PATH_INFO: `/v3/projects/352d9e66fb14400c8c46b34ae374a38b` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:36.892403 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c4e5a6-a6ba-40fb-8d5a-0c5a74e76445 None admin] RBAC: Authorizing `identity:get_project(project_id=352d9e66fb14400c8c46b34ae374a38b)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:36.893839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c4e5a6-a6ba-40fb-8d5a-0c5a74e76445 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:36.895596 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 584/1152] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:11:36 2022] GET /identity/v3/projects/352d9e66fb14400c8c46b34ae374a38b => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:36.901895 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f99129-4132-4082-8ed6-041e2fc9f303 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:37.006346 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f99129-4132-4082-8ed6-041e2fc9f303 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:37.041908 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e9de0-5c06-4419-8203-6a6778885cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:37.042880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e9de0-5c06-4419-8203-6a6778885cd9 None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:37.042880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e9de0-5c06-4419-8203-6a6778885cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:37.043149 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e9de0-5c06-4419-8203-6a6778885cd9 None admin] PATH_INFO: `/v3/projects/352d9e66fb14400c8c46b34ae374a38b/users/3e5cad8a50a44f78b8ab4e109fe7e0a5/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:37.045227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e9de0-5c06-4419-8203-6a6778885cd9 None admin] RBAC: Authorizing `identity:create_grant(project_id=352d9e66fb14400c8c46b34ae374a38b, user_id=3e5cad8a50a44f78b8ab4e109fe7e0a5, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:37.046656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e9de0-5c06-4419-8203-6a6778885cd9 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:37.125414 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 569/1153] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:11:36 2022] PUT /identity/v3/projects/352d9e66fb14400c8c46b34ae374a38b/users/3e5cad8a50a44f78b8ab4e109fe7e0a5/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:11:39.305122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e8fb1-7d2e-49b5-be93-4a088bd91483 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:39.305726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e8fb1-7d2e-49b5-be93-4a088bd91483 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:39.305726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e8fb1-7d2e-49b5-be93-4a088bd91483 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:39.307073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 585/1154] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:11:39.313844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb25d79-8b40-42eb-9bee-57e62ef9781c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:39.314397 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb25d79-8b40-42eb-9bee-57e62ef9781c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:39.314397 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb25d79-8b40-42eb-9bee-57e62ef9781c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:39.334214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-0cb25d79-8b40-42eb-9bee-57e62ef9781c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:39.335456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-0cb25d79-8b40-42eb-9bee-57e62ef9781c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:39.361062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:39.407964 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 570/1155] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:11:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:11:39.684290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e8fb1-7d2e-49b5-be93-4a088bd91483 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:39.697488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:39.792951 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e8fb1-7d2e-49b5-be93-4a088bd91483 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:39.830264 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ba4a3-edfe-4bc7-95ee-407a29ed8897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:39.830901 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ba4a3-edfe-4bc7-95ee-407a29ed8897 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:39.831206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ba4a3-edfe-4bc7-95ee-407a29ed8897 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:39.831478 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ba4a3-edfe-4bc7-95ee-407a29ed8897 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:39.842477 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ba4a3-edfe-4bc7-95ee-407a29ed8897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:11:39.847423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-f93ba4a3-edfe-4bc7-95ee-407a29ed8897 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:11:39.847423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:11:39.847423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:11:39.847423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:11:39.847423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:11:39.847423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:11:39.847423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:11:39.847423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:11:39.847423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:11:39.847423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:11:39.847423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:11:39.847423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:11:39.847423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:11:39.847423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:11:39.847423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:11:39.847423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:11:39.848597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:11:39.848597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:11:39.848597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:11:39.848597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:11:39.848597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:11:39.848597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:11:39.848597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:11:39.848597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:11:39.848597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:11:39.848597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:11:39.848597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:11:39.848597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:11:39.848597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:11:39.848597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:11:39.848597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:11:39.848597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:11:39.849844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:11:39.849844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:11:39.849844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:11:39.849844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:11:39.849844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:11:39.849844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:11:39.849844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:11:39.849844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:11:39.849844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:11:39.849844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:11:39.849844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:11:39.849844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:11:39.849844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 586/1156] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:11:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:11:39.855127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb25d79-8b40-42eb-9bee-57e62ef9781c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:39.970563 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb25d79-8b40-42eb-9bee-57e62ef9781c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:40.010223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d7825-0c99-40cf-bdbd-b24934cd14fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:40.011311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d7825-0c99-40cf-bdbd-b24934cd14fb None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:40.011819 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d7825-0c99-40cf-bdbd-b24934cd14fb None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:40.012262 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d7825-0c99-40cf-bdbd-b24934cd14fb None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:40.013478 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d7825-0c99-40cf-bdbd-b24934cd14fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:40.014030 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d7825-0c99-40cf-bdbd-b24934cd14fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:40.016692 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d7825-0c99-40cf-bdbd-b24934cd14fb None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:40.039248 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 571/1157] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:11:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:40.046962 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ba4a3-edfe-4bc7-95ee-407a29ed8897 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:40.124454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:40.158921 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ba4a3-edfe-4bc7-95ee-407a29ed8897 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:40.196032 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5712a-d122-40d7-84eb-54c229c1fb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:40.196668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5712a-d122-40d7-84eb-54c229c1fb0a None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:40.196948 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5712a-d122-40d7-84eb-54c229c1fb0a None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:40.197222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5712a-d122-40d7-84eb-54c229c1fb0a None admin] PATH_INFO: `/v3/users/3e5cad8a50a44f78b8ab4e109fe7e0a5` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:40.198538 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5712a-d122-40d7-84eb-54c229c1fb0a None admin] RBAC: Authorizing `identity:get_user(user_id=3e5cad8a50a44f78b8ab4e109fe7e0a5)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:40.201050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5712a-d122-40d7-84eb-54c229c1fb0a None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:40.202746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 587/1158] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:11:40 2022] GET /identity/v3/users/3e5cad8a50a44f78b8ab4e109fe7e0a5 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:40.209221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d7825-0c99-40cf-bdbd-b24934cd14fb None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:40.237073 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:40.323790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d7825-0c99-40cf-bdbd-b24934cd14fb None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:40.366286 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1c091-e7c0-4371-abe3-3cc254e10860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:40.366944 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1c091-e7c0-4371-abe3-3cc254e10860 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:40.367248 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1c091-e7c0-4371-abe3-3cc254e10860 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:40.367515 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1c091-e7c0-4371-abe3-3cc254e10860 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:40.450765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1c091-e7c0-4371-abe3-3cc254e10860 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:40.452277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1c091-e7c0-4371-abe3-3cc254e10860 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:40.477748 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:40.568022 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 572/1159] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:11:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:11:40.573454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5712a-d122-40d7-84eb-54c229c1fb0a None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:40.681700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5712a-d122-40d7-84eb-54c229c1fb0a None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:40.720574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a2b3c-4f91-43e6-92cb-7c96562e0b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:40.721220 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a2b3c-4f91-43e6-92cb-7c96562e0b38 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:40.721498 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a2b3c-4f91-43e6-92cb-7c96562e0b38 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:40.721767 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a2b3c-4f91-43e6-92cb-7c96562e0b38 None admin] PATH_INFO: `/v3/projects/352d9e66fb14400c8c46b34ae374a38b` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:40.722918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a2b3c-4f91-43e6-92cb-7c96562e0b38 None admin] RBAC: Authorizing `identity:get_project(project_id=352d9e66fb14400c8c46b34ae374a38b)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:40.724402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a2b3c-4f91-43e6-92cb-7c96562e0b38 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:40.726074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 588/1160] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:11:40 2022] GET /identity/v3/projects/352d9e66fb14400c8c46b34ae374a38b => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:11:40.733477 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1c091-e7c0-4371-abe3-3cc254e10860 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:40.842376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1c091-e7c0-4371-abe3-3cc254e10860 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:40.864800 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:40.880396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f674c-f646-4a0e-903b-24a84e04e81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:40.881029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f674c-f646-4a0e-903b-24a84e04e81d None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:40.881310 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f674c-f646-4a0e-903b-24a84e04e81d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:40.881588 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f674c-f646-4a0e-903b-24a84e04e81d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:40.882334 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f674c-f646-4a0e-903b-24a84e04e81d None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=352d9e66fb14400c8c46b34ae374a38b, user.id=3e5cad8a50a44f78b8ab4e109fe7e0a5) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:40.882642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f674c-f646-4a0e-903b-24a84e04e81d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:40.884174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f674c-f646-4a0e-903b-24a84e04e81d None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:40.891338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 573/1161] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:11:40 2022] GET /identity/v3/role_assignments?user.id=3e5cad8a50a44f78b8ab4e109fe7e0a5&scope.project.id=352d9e66fb14400c8c46b34ae374a38b&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:43.128442 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac79e5-de38-4022-9316-69893e9a8e68 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:43.128988 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac79e5-de38-4022-9316-69893e9a8e68 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:43.128988 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac79e5-de38-4022-9316-69893e9a8e68 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:43.130365 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 589/1162] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:11:43.137051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33567f3-4a24-45b9-b226-a3daa8be7073 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:43.137564 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33567f3-4a24-45b9-b226-a3daa8be7073 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:43.137795 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33567f3-4a24-45b9-b226-a3daa8be7073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:43.156736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-f33567f3-4a24-45b9-b226-a3daa8be7073 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:43.157959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f33567f3-4a24-45b9-b226-a3daa8be7073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:43.233563 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 574/1163] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:11:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:11:43.510456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac79e5-de38-4022-9316-69893e9a8e68 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:43.520772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:43.621594 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac79e5-de38-4022-9316-69893e9a8e68 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:43.656048 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463bd89-f188-453e-ab79-ea0657d1a84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:43.657698 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463bd89-f188-453e-ab79-ea0657d1a84c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:43.657698 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463bd89-f188-453e-ab79-ea0657d1a84c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:43.657698 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463bd89-f188-453e-ab79-ea0657d1a84c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:43.726773 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7463bd89-f188-453e-ab79-ea0657d1a84c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:43.728251 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7463bd89-f188-453e-ab79-ea0657d1a84c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:43.736993 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:43.830769 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 590/1164] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:11:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:11:43.836832 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33567f3-4a24-45b9-b226-a3daa8be7073 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:43.938499 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33567f3-4a24-45b9-b226-a3daa8be7073 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:43.952829 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:43.973075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d07f71-a277-4f70-bb4b-5eff94c58ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:43.973710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07f71-a277-4f70-bb4b-5eff94c58ef5 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:43.978739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07f71-a277-4f70-bb4b-5eff94c58ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:43.978739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07f71-a277-4f70-bb4b-5eff94c58ef5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:43.981943 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d07f71-a277-4f70-bb4b-5eff94c58ef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:43.983270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d07f71-a277-4f70-bb4b-5eff94c58ef5 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:43.994506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-85d07f71-a277-4f70-bb4b-5eff94c58ef5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 07 13:11:43.994506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:11:43.994506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 07 13:11:43.994506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 07 13:11:43.994506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:11:43.994506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:11:43.994506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:11:43.994506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:11:43.994506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 07 13:11:43.994506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:11:43.994506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:11:43.994506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:11:43.994506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:11:43.994506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:11:43.994506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:11:43.994506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:11:43.994506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:11:43.995803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:11:43.995803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:11:43.995803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:11:43.995803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:11:43.995803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:11:43.995803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:11:43.995803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:11:43.995803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:11:43.995803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:11:43.995803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:11:43.995803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:11:43.995803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:11:43.995803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:11:43.995803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:11:43.995803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:11:43.995803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:11:43.997967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:11:43.997967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:11:43.997967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:11:43.997967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:11:43.997967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:11:43.997967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:11:43.997967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:11:43.997967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:11:43.997967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:11:43.997967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:11:43.997967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 07 13:11:43.997967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:11:43.997967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 07 13:11:43.997967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:11:43.997967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 575/1165] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 7 13:11:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:11:44.002655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463bd89-f188-453e-ab79-ea0657d1a84c None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:44.101747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463bd89-f188-453e-ab79-ea0657d1a84c None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:44.110077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:44.136940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8ebbd-400d-495e-8e29-9c26c021000c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:44.137674 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8ebbd-400d-495e-8e29-9c26c021000c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:44.137822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8ebbd-400d-495e-8e29-9c26c021000c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:44.138074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8ebbd-400d-495e-8e29-9c26c021000c None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:44.138720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8ebbd-400d-495e-8e29-9c26c021000c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:44.139051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8ebbd-400d-495e-8e29-9c26c021000c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:44.140471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8ebbd-400d-495e-8e29-9c26c021000c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:44.148401 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 591/1166] 10.222.0.62 () {56 vars in 1115 bytes} [Wed Sep 7 13:11:44 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:46.363374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709f6d5-e3fc-4671-9e50-c687fba0f037 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:46.363960 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709f6d5-e3fc-4671-9e50-c687fba0f037 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:46.363960 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709f6d5-e3fc-4671-9e50-c687fba0f037 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:46.365271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 576/1167] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:11:46.370883 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52988cc0-a8b3-4b19-828d-a8ea6bcfa7d5 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:46.371935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52988cc0-a8b3-4b19-828d-a8ea6bcfa7d5 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:46.372223 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52988cc0-a8b3-4b19-828d-a8ea6bcfa7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:46.392225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-52988cc0-a8b3-4b19-828d-a8ea6bcfa7d5 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:46.393440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-52988cc0-a8b3-4b19-828d-a8ea6bcfa7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:46.466553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 592/1168] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:11:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:11:46.745791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709f6d5-e3fc-4671-9e50-c687fba0f037 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:46.861380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709f6d5-e3fc-4671-9e50-c687fba0f037 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:46.898076 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a224a88-a2ac-4aa0-b011-06cdc1e3c539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:46.898690 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a224a88-a2ac-4aa0-b011-06cdc1e3c539 None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:46.898997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a224a88-a2ac-4aa0-b011-06cdc1e3c539 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:46.899270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a224a88-a2ac-4aa0-b011-06cdc1e3c539 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:46.899974 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a224a88-a2ac-4aa0-b011-06cdc1e3c539 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:46.901342 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a224a88-a2ac-4aa0-b011-06cdc1e3c539 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:46.989893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 577/1169] 10.222.0.62 () {60 vars in 1138 bytes} [Wed Sep 7 13:11:46 2022] POST /identity/v3/domains => generated 267 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:11:49.245796 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebae028-4d91-49a5-9d16-1794c301f4ce None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:49.246516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebae028-4d91-49a5-9d16-1794c301f4ce None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:49.247306 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebae028-4d91-49a5-9d16-1794c301f4ce None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:49.248696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 593/1170] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:11:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:11:49.254594 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f88c5-080c-438c-931c-cf0df98d8909 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:49.255155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f88c5-080c-438c-931c-cf0df98d8909 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:49.255155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f88c5-080c-438c-931c-cf0df98d8909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:49.274882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-2d8f88c5-080c-438c-931c-cf0df98d8909 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:49.276214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2d8f88c5-080c-438c-931c-cf0df98d8909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:49.306268 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:49.348299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 578/1171] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:11:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:11:49.629577 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebae028-4d91-49a5-9d16-1794c301f4ce None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:49.737238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:49.741259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebae028-4d91-49a5-9d16-1794c301f4ce None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:49.778459 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330fb238-6c4e-4d77-8aa9-a474a9e1b4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:49.779128 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fb238-6c4e-4d77-8aa9-a474a9e1b4ab None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:49.779414 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fb238-6c4e-4d77-8aa9-a474a9e1b4ab None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:49.779688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fb238-6c4e-4d77-8aa9-a474a9e1b4ab None admin] PATH_INFO: `/v3/domains/2cdb690dd7924d2e96781e7ab060cc02` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:49.790571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330fb238-6c4e-4d77-8aa9-a474a9e1b4ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=2cdb690dd7924d2e96781e7ab060cc02)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:49.792058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330fb238-6c4e-4d77-8aa9-a474a9e1b4ab None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:49.794517 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 594/1172] 10.222.0.62 () {56 vars in 1183 bytes} [Wed Sep 7 13:11:49 2022] GET /identity/v3/domains/2cdb690dd7924d2e96781e7ab060cc02 => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:49.800606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f88c5-080c-438c-931c-cf0df98d8909 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:49.905325 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f88c5-080c-438c-931c-cf0df98d8909 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:49.940440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51706f-7ed3-46d9-856b-325a261eddea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:49.941050 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51706f-7ed3-46d9-856b-325a261eddea None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:49.941331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51706f-7ed3-46d9-856b-325a261eddea None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:49.941611 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51706f-7ed3-46d9-856b-325a261eddea None admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:49.942274 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51706f-7ed3-46d9-856b-325a261eddea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:49.943720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51706f-7ed3-46d9-856b-325a261eddea None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:50.053843 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 579/1173] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 7 13:11:49 2022] POST /identity/v3/projects => generated 361 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:11:52.272548 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cd20f-253f-4102-8f06-f9dfa1508b15 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:52.273116 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cd20f-253f-4102-8f06-f9dfa1508b15 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:52.273116 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cd20f-253f-4102-8f06-f9dfa1508b15 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:52.274460 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 595/1174] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:11:52.280911 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7add5c3-cc63-4b4d-9d50-1f0bb2e21fce None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:52.281431 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7add5c3-cc63-4b4d-9d50-1f0bb2e21fce None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:52.281526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7add5c3-cc63-4b4d-9d50-1f0bb2e21fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:52.299871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c7add5c3-cc63-4b4d-9d50-1f0bb2e21fce None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:52.301062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c7add5c3-cc63-4b4d-9d50-1f0bb2e21fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:52.312327 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:52.368034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 580/1175] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:11:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:11:52.645523 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552cd20f-253f-4102-8f06-f9dfa1508b15 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:52.748820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552cd20f-253f-4102-8f06-f9dfa1508b15 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:52.782009 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b429e-48cb-47dd-8a76-63396afc4ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:52.782622 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b429e-48cb-47dd-8a76-63396afc4ab2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:52.782921 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b429e-48cb-47dd-8a76-63396afc4ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:52.783215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b429e-48cb-47dd-8a76-63396afc4ab2 None admin] PATH_INFO: `/v3/domains/2cdb690dd7924d2e96781e7ab060cc02` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:52.784325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b429e-48cb-47dd-8a76-63396afc4ab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2cdb690dd7924d2e96781e7ab060cc02)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:52.785723 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b429e-48cb-47dd-8a76-63396afc4ab2 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:52.787326 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 596/1176] 10.222.0.62 () {56 vars in 1183 bytes} [Wed Sep 7 13:11:52 2022] GET /identity/v3/domains/2cdb690dd7924d2e96781e7ab060cc02 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:52.793685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7add5c3-cc63-4b4d-9d50-1f0bb2e21fce None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:52.893492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7add5c3-cc63-4b4d-9d50-1f0bb2e21fce None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:52.926888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff00661-4ffb-405d-81c8-162a35464bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:52.927446 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff00661-4ffb-405d-81c8-162a35464bc1 None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:52.927725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff00661-4ffb-405d-81c8-162a35464bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:52.927994 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff00661-4ffb-405d-81c8-162a35464bc1 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:52.928657 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff00661-4ffb-405d-81c8-162a35464bc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:52.930082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff00661-4ffb-405d-81c8-162a35464bc1 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:53.058024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 581/1177] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:11:52 2022] POST /identity/v3/users => generated 311 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:11:55.282685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f26557-843e-48fa-a4a4-519e97c503f3 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:55.283284 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f26557-843e-48fa-a4a4-519e97c503f3 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:55.283284 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f26557-843e-48fa-a4a4-519e97c503f3 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:55.285000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 597/1178] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:11:55.290666 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeffccc-2874-4794-a2bb-0d671fa4dc29 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:55.291132 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeffccc-2874-4794-a2bb-0d671fa4dc29 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:55.291241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeffccc-2874-4794-a2bb-0d671fa4dc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:55.297696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:55.311012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-6eeffccc-2874-4794-a2bb-0d671fa4dc29 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:55.319430 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6eeffccc-2874-4794-a2bb-0d671fa4dc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:55.380637 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 582/1179] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:11:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:11:55.651261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f26557-843e-48fa-a4a4-519e97c503f3 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:55.705690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:55.793525 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f26557-843e-48fa-a4a4-519e97c503f3 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:55.829513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945f174-f477-4e31-8718-3bd5493ec99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:55.830143 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945f174-f477-4e31-8718-3bd5493ec99a None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:55.830419 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945f174-f477-4e31-8718-3bd5493ec99a None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:55.830686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945f174-f477-4e31-8718-3bd5493ec99a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:55.841776 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945f174-f477-4e31-8718-3bd5493ec99a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:11:55.846602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-4945f174-f477-4e31-8718-3bd5493ec99a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:11:55.846602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:11:55.846602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:11:55.846602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:11:55.846602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:11:55.846602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:11:55.846602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:11:55.846602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:11:55.846602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:11:55.846602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:11:55.846602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:11:55.846602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:11:55.846602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:11:55.846602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:11:55.846602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:11:55.846602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:11:55.847774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:11:55.847774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:11:55.847774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:11:55.847774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:11:55.847774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:11:55.847774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:11:55.847774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:11:55.847774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:11:55.847774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:11:55.847774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:11:55.847774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:11:55.847774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:11:55.847774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:11:55.847774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:11:55.847774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:11:55.847774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:11:55.849023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:11:55.849023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:11:55.849023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:11:55.849023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:11:55.849023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:11:55.849023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:11:55.849023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:11:55.849023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:11:55.849023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:11:55.849023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:11:55.849023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:11:55.849023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:11:55.849023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 598/1180] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:11:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:11:55.853995 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeffccc-2874-4794-a2bb-0d671fa4dc29 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:55.960933 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeffccc-2874-4794-a2bb-0d671fa4dc29 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:55.992049 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:55.997650 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a86c3-06e1-4bf9-9ac5-2d9dccfb810b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:55.998267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a86c3-06e1-4bf9-9ac5-2d9dccfb810b None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:55.998548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a86c3-06e1-4bf9-9ac5-2d9dccfb810b None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:55.999792 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a86c3-06e1-4bf9-9ac5-2d9dccfb810b None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:55.999792 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a86c3-06e1-4bf9-9ac5-2d9dccfb810b None admin] RBAC: Adding query filter params (name=admin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:55.999980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a86c3-06e1-4bf9-9ac5-2d9dccfb810b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:56.001267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a86c3-06e1-4bf9-9ac5-2d9dccfb810b None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:56.009966 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 583/1181] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:11:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:56.017440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945f174-f477-4e31-8718-3bd5493ec99a None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:56.116445 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:56.129567 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945f174-f477-4e31-8718-3bd5493ec99a None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:56.167683 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b571edb-7233-4397-95c1-886b95556b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:56.168322 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b571edb-7233-4397-95c1-886b95556b6f None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:56.168601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b571edb-7233-4397-95c1-886b95556b6f None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:56.168872 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b571edb-7233-4397-95c1-886b95556b6f None admin] PATH_INFO: `/v3/users/e892608131e2434785a577033c749e3e` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:56.184240 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b571edb-7233-4397-95c1-886b95556b6f None admin] RBAC: Authorizing `identity:get_user(user_id=e892608131e2434785a577033c749e3e)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:56.185899 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b571edb-7233-4397-95c1-886b95556b6f None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:56.187680 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 599/1182] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:11:56 2022] GET /identity/v3/users/e892608131e2434785a577033c749e3e => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:56.193748 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a86c3-06e1-4bf9-9ac5-2d9dccfb810b None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:56.302210 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a86c3-06e1-4bf9-9ac5-2d9dccfb810b None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:56.337431 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da212ebe-d0d9-4118-b3bb-744f3adf862b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:56.338041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da212ebe-d0d9-4118-b3bb-744f3adf862b None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:56.338319 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da212ebe-d0d9-4118-b3bb-744f3adf862b None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:56.338588 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da212ebe-d0d9-4118-b3bb-744f3adf862b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:56.399343 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:56.407705 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da212ebe-d0d9-4118-b3bb-744f3adf862b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:56.411509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da212ebe-d0d9-4118-b3bb-744f3adf862b None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:56.509086 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 584/1183] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:11:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:11:56.515497 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b571edb-7233-4397-95c1-886b95556b6f None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:56.624777 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b571edb-7233-4397-95c1-886b95556b6f None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:56.666627 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63f7a4-4049-4174-b81f-b665d6648ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:56.667329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63f7a4-4049-4174-b81f-b665d6648ab4 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:56.667671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63f7a4-4049-4174-b81f-b665d6648ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:56.667946 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63f7a4-4049-4174-b81f-b665d6648ab4 None admin] PATH_INFO: `/v3/projects/94e4d7c578cf48799d3e10ca85c6a461` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:56.669301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63f7a4-4049-4174-b81f-b665d6648ab4 None admin] RBAC: Authorizing `identity:get_project(project_id=94e4d7c578cf48799d3e10ca85c6a461)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:56.670810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63f7a4-4049-4174-b81f-b665d6648ab4 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:56.672633 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 600/1184] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:11:56 2022] GET /identity/v3/projects/94e4d7c578cf48799d3e10ca85c6a461 => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:11:56.680792 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da212ebe-d0d9-4118-b3bb-744f3adf862b None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:56.789294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:56.796678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da212ebe-d0d9-4118-b3bb-744f3adf862b None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:56.834322 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd8ca5-5006-480f-a807-202d8147166b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:56.835618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd8ca5-5006-480f-a807-202d8147166b None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:56.836052 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd8ca5-5006-480f-a807-202d8147166b None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:56.836168 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd8ca5-5006-480f-a807-202d8147166b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:56.836892 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd8ca5-5006-480f-a807-202d8147166b None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=94e4d7c578cf48799d3e10ca85c6a461, user.id=e892608131e2434785a577033c749e3e) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:56.837242 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd8ca5-5006-480f-a807-202d8147166b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:56.838902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd8ca5-5006-480f-a807-202d8147166b None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:56.845497 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 585/1185] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:11:56 2022] GET /identity/v3/role_assignments?user.id=e892608131e2434785a577033c749e3e&scope.project.id=94e4d7c578cf48799d3e10ca85c6a461&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:58.998881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f33337-1e74-40ff-a26f-e5699dd73fb1 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:58.999483 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f33337-1e74-40ff-a26f-e5699dd73fb1 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:58.999594 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f33337-1e74-40ff-a26f-e5699dd73fb1 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:59.000916 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 601/1186] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:11:59.006318 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b50c37-36a1-4740-a5f7-a7ed83c340ab None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:59.006764 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b50c37-36a1-4740-a5f7-a7ed83c340ab None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:59.006853 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b50c37-36a1-4740-a5f7-a7ed83c340ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:59.024855 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-74b50c37-36a1-4740-a5f7-a7ed83c340ab None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:11:59.026038 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-74b50c37-36a1-4740-a5f7-a7ed83c340ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:11:59.094020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 586/1187] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:11:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:11:59.363960 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f33337-1e74-40ff-a26f-e5699dd73fb1 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:59.380355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:59.469561 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f33337-1e74-40ff-a26f-e5699dd73fb1 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:59.504430 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2e5d8-6f33-4c69-916a-73346de79d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:59.505057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2e5d8-6f33-4c69-916a-73346de79d22 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:59.505335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2e5d8-6f33-4c69-916a-73346de79d22 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:59.505606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2e5d8-6f33-4c69-916a-73346de79d22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:59.515893 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2e5d8-6f33-4c69-916a-73346de79d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:11:59.520675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-c7a2e5d8-6f33-4c69-916a-73346de79d22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:11:59.520675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:11:59.520675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:11:59.520675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:11:59.520675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:11:59.520675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:11:59.520675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:11:59.520675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:11:59.520675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:11:59.520675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:11:59.520675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:11:59.520675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:11:59.520675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:11:59.520675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:11:59.520675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:11:59.520675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:11:59.521794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:11:59.521794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:11:59.521794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:11:59.521794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:11:59.521794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:11:59.521794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:11:59.521794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:11:59.521794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:11:59.521794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:11:59.521794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:11:59.521794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:11:59.521794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:11:59.521794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:11:59.521794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:11:59.521794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:11:59.521794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:11:59.522787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:11:59.522787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:11:59.522787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:11:59.522787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:11:59.522787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:11:59.522787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:11:59.522787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:11:59.522787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:11:59.522787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:11:59.522787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:11:59.522787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:11:59.522787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:11:59.522787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 602/1188] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:11:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:11:59.527971 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b50c37-36a1-4740-a5f7-a7ed83c340ab None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:59.615898 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:59.628258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b50c37-36a1-4740-a5f7-a7ed83c340ab None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:59.661162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf14789-4927-4eaf-b4b0-c95027c95566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:59.661771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf14789-4927-4eaf-b4b0-c95027c95566 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:59.662054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf14789-4927-4eaf-b4b0-c95027c95566 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:59.662328 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf14789-4927-4eaf-b4b0-c95027c95566 None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:59.663033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf14789-4927-4eaf-b4b0-c95027c95566 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:11:59.663342 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf14789-4927-4eaf-b4b0-c95027c95566 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:59.664699 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf14789-4927-4eaf-b4b0-c95027c95566 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:59.677241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 587/1189] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:11:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:59.684281 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2e5d8-6f33-4c69-916a-73346de79d22 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:59.761829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:11:59.787497 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2e5d8-6f33-4c69-916a-73346de79d22 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:59.822946 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a308e8-2a86-4c53-b7ff-dda8dc89fee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:59.823713 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a308e8-2a86-4c53-b7ff-dda8dc89fee8 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:59.824571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a308e8-2a86-4c53-b7ff-dda8dc89fee8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:59.824864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a308e8-2a86-4c53-b7ff-dda8dc89fee8 None admin] PATH_INFO: `/v3/users/e892608131e2434785a577033c749e3e` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:59.826072 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a308e8-2a86-4c53-b7ff-dda8dc89fee8 None admin] RBAC: Authorizing `identity:get_user(user_id=e892608131e2434785a577033c749e3e)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:59.827545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a308e8-2a86-4c53-b7ff-dda8dc89fee8 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:59.829476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 603/1190] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:11:59 2022] GET /identity/v3/users/e892608131e2434785a577033c749e3e => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:59.835489 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf14789-4927-4eaf-b4b0-c95027c95566 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:11:59.941604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf14789-4927-4eaf-b4b0-c95027c95566 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:11:59.976888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90c58f-b420-4c73-8e4a-a41b9d260012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:11:59.977534 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90c58f-b420-4c73-8e4a-a41b9d260012 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:11:59.977813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90c58f-b420-4c73-8e4a-a41b9d260012 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:11:59.978085 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90c58f-b420-4c73-8e4a-a41b9d260012 None admin] PATH_INFO: `/v3/projects/94e4d7c578cf48799d3e10ca85c6a461` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:11:59.979233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90c58f-b420-4c73-8e4a-a41b9d260012 None admin] RBAC: Authorizing `identity:get_project(project_id=94e4d7c578cf48799d3e10ca85c6a461)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:11:59.981469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90c58f-b420-4c73-8e4a-a41b9d260012 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:11:59.983264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 588/1191] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:11:59 2022] GET /identity/v3/projects/94e4d7c578cf48799d3e10ca85c6a461 => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:11:59.989283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a308e8-2a86-4c53-b7ff-dda8dc89fee8 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:00.101948 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a308e8-2a86-4c53-b7ff-dda8dc89fee8 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:00.140129 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c7134-d82c-40d5-8771-707a07ef4b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:00.140804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c7134-d82c-40d5-8771-707a07ef4b45 None admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:00.141101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c7134-d82c-40d5-8771-707a07ef4b45 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:00.141353 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c7134-d82c-40d5-8771-707a07ef4b45 None admin] PATH_INFO: `/v3/projects/94e4d7c578cf48799d3e10ca85c6a461/users/e892608131e2434785a577033c749e3e/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:00.143331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c7134-d82c-40d5-8771-707a07ef4b45 None admin] RBAC: Authorizing `identity:create_grant(project_id=94e4d7c578cf48799d3e10ca85c6a461, user_id=e892608131e2434785a577033c749e3e, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:00.144765 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c7134-d82c-40d5-8771-707a07ef4b45 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:00.286965 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 604/1192] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:11:59 2022] PUT /identity/v3/projects/94e4d7c578cf48799d3e10ca85c6a461/users/e892608131e2434785a577033c749e3e/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:12:02.455475 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db291ec6-99bf-4403-bcb4-2e2b03a0e15a None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:02.456011 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db291ec6-99bf-4403-bcb4-2e2b03a0e15a None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:02.456011 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db291ec6-99bf-4403-bcb4-2e2b03a0e15a None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:02.457343 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 589/1193] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:12:02.464088 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3be62-6c29-4463-8817-9e99339d5618 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:02.464618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3be62-6c29-4463-8817-9e99339d5618 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:02.465349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3be62-6c29-4463-8817-9e99339d5618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:02.473584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:12:02.487318 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-98c3be62-6c29-4463-8817-9e99339d5618 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:12:02.498045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-98c3be62-6c29-4463-8817-9e99339d5618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:12:02.556646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 605/1194] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:12:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:12:02.828766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db291ec6-99bf-4403-bcb4-2e2b03a0e15a None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:02.849874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:12:02.938237 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db291ec6-99bf-4403-bcb4-2e2b03a0e15a None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:02.974830 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508478b-bd15-49ff-9ebf-e28ea208dba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:02.975491 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508478b-bd15-49ff-9ebf-e28ea208dba1 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:02.975781 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508478b-bd15-49ff-9ebf-e28ea208dba1 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:02.976057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508478b-bd15-49ff-9ebf-e28ea208dba1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:02.987617 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508478b-bd15-49ff-9ebf-e28ea208dba1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:12:02.992465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-e508478b-bd15-49ff-9ebf-e28ea208dba1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:12:02.992465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:12:02.992465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:12:02.992465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:12:02.992465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:12:02.992465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:12:02.992465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:12:02.992465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:12:02.992465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:12:02.992465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:12:02.992465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:12:02.992465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:12:02.992465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:12:02.992465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:12:02.992465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:12:02.992465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:12:02.993612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:12:02.993612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:12:02.993612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:12:02.993612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:12:02.993612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:12:02.993612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:12:02.993612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:12:02.993612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:12:02.993612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:12:02.993612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:12:02.993612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:12:02.993612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:12:02.993612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:12:02.993612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:12:02.993612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:12:02.993612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:12:02.994874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:12:02.994874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:12:02.994874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:12:02.994874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:12:02.994874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:12:02.994874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:12:02.994874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:12:02.994874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:12:02.994874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:12:02.994874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:12:02.994874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:12:02.994874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:12:02.994874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 590/1195] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:12:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:12:02.999970 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3be62-6c29-4463-8817-9e99339d5618 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:03.103772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3be62-6c29-4463-8817-9e99339d5618 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:03.133894 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:12:03.139562 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7154a-a34d-4531-b3ef-e0bb49101d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:03.140176 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7154a-a34d-4531-b3ef-e0bb49101d5d None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:03.140455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7154a-a34d-4531-b3ef-e0bb49101d5d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:03.140727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7154a-a34d-4531-b3ef-e0bb49101d5d None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:03.141393 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7154a-a34d-4531-b3ef-e0bb49101d5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:12:03.141700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7154a-a34d-4531-b3ef-e0bb49101d5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:03.143072 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7154a-a34d-4531-b3ef-e0bb49101d5d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:03.151572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 606/1196] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:12:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:12:03.158249 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508478b-bd15-49ff-9ebf-e28ea208dba1 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:03.241997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:12:03.264552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508478b-bd15-49ff-9ebf-e28ea208dba1 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:03.301633 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892a996-a182-4487-8b61-27c1239b0442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:03.302270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892a996-a182-4487-8b61-27c1239b0442 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:03.302545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892a996-a182-4487-8b61-27c1239b0442 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:03.302812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892a996-a182-4487-8b61-27c1239b0442 None admin] PATH_INFO: `/v3/users/e892608131e2434785a577033c749e3e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:03.303977 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892a996-a182-4487-8b61-27c1239b0442 None admin] RBAC: Authorizing `identity:get_user(user_id=e892608131e2434785a577033c749e3e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:03.305426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892a996-a182-4487-8b61-27c1239b0442 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:03.307198 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 591/1197] 10.222.0.62 () {56 vars in 1177 bytes} [Wed Sep 7 13:12:03 2022] GET /identity/v3/users/e892608131e2434785a577033c749e3e => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:12:03.313238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7154a-a34d-4531-b3ef-e0bb49101d5d None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:03.418026 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7154a-a34d-4531-b3ef-e0bb49101d5d None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:03.452152 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8125d0-01d6-455b-b281-fdce335ca9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:03.453008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8125d0-01d6-455b-b281-fdce335ca9af None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:03.453287 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8125d0-01d6-455b-b281-fdce335ca9af None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:03.453567 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8125d0-01d6-455b-b281-fdce335ca9af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:03.514805 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:12:03.522794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8125d0-01d6-455b-b281-fdce335ca9af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:03.524297 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8125d0-01d6-455b-b281-fdce335ca9af None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:03.631121 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 607/1198] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:12:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:12:03.637856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892a996-a182-4487-8b61-27c1239b0442 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:03.740262 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892a996-a182-4487-8b61-27c1239b0442 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:03.776346 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a7ff3-5f89-4396-a401-48aa8e4271b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:03.777026 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a7ff3-5f89-4396-a401-48aa8e4271b9 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:03.777303 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a7ff3-5f89-4396-a401-48aa8e4271b9 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:03.777574 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a7ff3-5f89-4396-a401-48aa8e4271b9 None admin] PATH_INFO: `/v3/projects/94e4d7c578cf48799d3e10ca85c6a461` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:03.778711 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a7ff3-5f89-4396-a401-48aa8e4271b9 None admin] RBAC: Authorizing `identity:get_project(project_id=94e4d7c578cf48799d3e10ca85c6a461)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:03.780213 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a7ff3-5f89-4396-a401-48aa8e4271b9 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:03.781880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 592/1199] 10.222.0.62 () {56 vars in 1186 bytes} [Wed Sep 7 13:12:03 2022] GET /identity/v3/projects/94e4d7c578cf48799d3e10ca85c6a461 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:12:03.788847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8125d0-01d6-455b-b281-fdce335ca9af None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:12:03.882869 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:12:03.890406 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8125d0-01d6-455b-b281-fdce335ca9af None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:12:03.926191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067848d4-81b8-41db-970b-f9b74d670225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:12:03.926792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067848d4-81b8-41db-970b-f9b74d670225 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:12:03.927204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067848d4-81b8-41db-970b-f9b74d670225 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:12:03.927365 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067848d4-81b8-41db-970b-f9b74d670225 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:12:03.928123 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067848d4-81b8-41db-970b-f9b74d670225 None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=94e4d7c578cf48799d3e10ca85c6a461, user.id=e892608131e2434785a577033c749e3e) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:12:03.928436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067848d4-81b8-41db-970b-f9b74d670225 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:12:03.929859 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067848d4-81b8-41db-970b-f9b74d670225 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:12:03.937317 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 608/1200] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:12:03 2022] GET /identity/v3/role_assignments?user.id=e892608131e2434785a577033c749e3e&scope.project.id=94e4d7c578cf48799d3e10ca85c6a461&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:12:13.771987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=93 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:12:13.933649 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=21 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:29.084713 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30541c9-7fc3-474c-a49b-b01c3da6a16c None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:29.085419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30541c9-7fc3-474c-a49b-b01c3da6a16c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:29.085419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30541c9-7fc3-474c-a49b-b01c3da6a16c None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:29.086917 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 593/1201] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:13:29.093095 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e589e-14ef-437c-8be1-fcac9469ae0e None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:29.094007 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e589e-14ef-437c-8be1-fcac9469ae0e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:29.094269 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e589e-14ef-437c-8be1-fcac9469ae0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:29.114836 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-3b2e589e-14ef-437c-8be1-fcac9469ae0e None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:13:29.116071 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-3b2e589e-14ef-437c-8be1-fcac9469ae0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:13:29.193614 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 609/1202] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:13:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:13:29.477332 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30541c9-7fc3-474c-a49b-b01c3da6a16c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:29.618823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30541c9-7fc3-474c-a49b-b01c3da6a16c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:29.659179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aed9e8-6bab-4dc4-a322-685b9ae64c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:29.660027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aed9e8-6bab-4dc4-a322-685b9ae64c6a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:29.660212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aed9e8-6bab-4dc4-a322-685b9ae64c6a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:29.660529 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aed9e8-6bab-4dc4-a322-685b9ae64c6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:29.674572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aed9e8-6bab-4dc4-a322-685b9ae64c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:29.676091 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aed9e8-6bab-4dc4-a322-685b9ae64c6a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:29.694112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-a6aed9e8-6bab-4dc4-a322-685b9ae64c6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:29.694112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:29.694112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:29.694112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:29.694112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:29.694112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:29.694112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:29.694112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:29.694112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:29.694112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:29.694112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:29.694112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:29.694112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:13:29.694112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:13:29.694112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:13:29.694112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:13:29.695945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:29.695945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:29.695945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:29.695945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:29.695945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:29.695945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:29.695945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:29.695945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:29.695945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:29.695945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:29.695945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:29.695945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:29.695945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:29.695945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:29.695945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:29.695945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:29.695945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:13:29.697221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:13:29.697221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:29.697221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:13:29.697221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 594/1203] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:29.702828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e589e-14ef-437c-8be1-fcac9469ae0e None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:29.858808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e589e-14ef-437c-8be1-fcac9469ae0e None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:29.898748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:29.909423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3fe463-eaec-4846-9fcc-056a84afc23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:29.910447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3fe463-eaec-4846-9fcc-056a84afc23e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:29.910924 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3fe463-eaec-4846-9fcc-056a84afc23e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:29.911493 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3fe463-eaec-4846-9fcc-056a84afc23e None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:29.912655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3fe463-eaec-4846-9fcc-056a84afc23e None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:29.913277 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3fe463-eaec-4846-9fcc-056a84afc23e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:29.915636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3fe463-eaec-4846-9fcc-056a84afc23e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:29.932628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 610/1204] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:13:29.942863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aed9e8-6bab-4dc4-a322-685b9ae64c6a None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:30.023037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:30.089754 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aed9e8-6bab-4dc4-a322-685b9ae64c6a None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:30.130483 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694af2a6-18cd-4e51-9588-3f33a660fa47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:30.131030 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694af2a6-18cd-4e51-9588-3f33a660fa47 None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:30.131309 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694af2a6-18cd-4e51-9588-3f33a660fa47 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:30.131576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694af2a6-18cd-4e51-9588-3f33a660fa47 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:30.132223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694af2a6-18cd-4e51-9588-3f33a660fa47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:30.133657 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694af2a6-18cd-4e51-9588-3f33a660fa47 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:30.337502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 595/1205] 10.222.0.62 () {60 vars in 1132 bytes} [Wed Sep 7 13:13:29 2022] POST /identity/v3/users => generated 251 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:13:32.850462 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0fc18-97bc-4759-a1ec-2d584aadc6b9 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:32.851079 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0fc18-97bc-4759-a1ec-2d584aadc6b9 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:32.851079 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0fc18-97bc-4759-a1ec-2d584aadc6b9 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:32.852380 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 611/1206] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:13:32.858470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d22ed-3ba0-43ba-af07-88899b1708c1 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:32.859020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d22ed-3ba0-43ba-af07-88899b1708c1 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:32.859020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d22ed-3ba0-43ba-af07-88899b1708c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:32.877745 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c81d22ed-3ba0-43ba-af07-88899b1708c1 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:13:32.878953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c81d22ed-3ba0-43ba-af07-88899b1708c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:13:32.957533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 596/1207] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:13:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:13:33.234210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0fc18-97bc-4759-a1ec-2d584aadc6b9 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:33.381002 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0fc18-97bc-4759-a1ec-2d584aadc6b9 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:33.436823 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17537dea-befb-4e56-aa5c-deebe16535c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:33.437468 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17537dea-befb-4e56-aa5c-deebe16535c5 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:33.437733 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17537dea-befb-4e56-aa5c-deebe16535c5 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:33.438003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17537dea-befb-4e56-aa5c-deebe16535c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:33.455252 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17537dea-befb-4e56-aa5c-deebe16535c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:13:33.464947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-17537dea-befb-4e56-aa5c-deebe16535c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:13:33.464947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:33.464947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:33.464947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:33.464947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:33.464947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:33.464947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:33.464947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:33.464947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:33.464947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:33.464947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:33.464947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:33.464947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:13:33.464947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:13:33.464947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:13:33.464947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:13:33.466255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:13:33.466255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:13:33.466255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:13:33.466255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:13:33.466255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:33.466255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:33.466255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:33.466255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:33.466255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:33.466255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:33.466255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:33.466255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:33.466255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:33.466255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:33.466255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:33.466255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:33.471481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:33.471481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:33.471481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:33.471481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:33.471481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:13:33.471481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:13:33.471481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:13:33.471481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:13:33.471481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:13:33.471481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:13:33.471481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:13:33.471481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:13:33.471481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 612/1208] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:13:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:33.472992 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d22ed-3ba0-43ba-af07-88899b1708c1 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:33.518901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:33.585190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d22ed-3ba0-43ba-af07-88899b1708c1 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:33.627317 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3f599-258b-46c9-80bb-9e7b06a64a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:33.628040 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3f599-258b-46c9-80bb-9e7b06a64a7c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:33.628329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3f599-258b-46c9-80bb-9e7b06a64a7c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:33.628604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3f599-258b-46c9-80bb-9e7b06a64a7c None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:33.629286 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3f599-258b-46c9-80bb-9e7b06a64a7c None admin] RBAC: Adding query filter params (name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:33.629591 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3f599-258b-46c9-80bb-9e7b06a64a7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:33.631020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3f599-258b-46c9-80bb-9e7b06a64a7c None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:33.641644 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 597/1209] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:13:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:33.648260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17537dea-befb-4e56-aa5c-deebe16535c5 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:33.723499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:33.809158 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17537dea-befb-4e56-aa5c-deebe16535c5 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:33.852772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec958e39-a367-4cf3-8741-1db97244f104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:33.853842 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec958e39-a367-4cf3-8741-1db97244f104 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:33.854300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec958e39-a367-4cf3-8741-1db97244f104 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:33.854736 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec958e39-a367-4cf3-8741-1db97244f104 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:33.874612 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec958e39-a367-4cf3-8741-1db97244f104 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:33.877388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec958e39-a367-4cf3-8741-1db97244f104 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:33.906619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-ec958e39-a367-4cf3-8741-1db97244f104 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:33.906619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:33.906619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:33.906619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:33.906619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:33.906619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:33.906619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:33.906619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:33.906619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:33.906619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:33.906619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:33.906619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:33.906619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:13:33.906619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:13:33.906619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:13:33.906619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:13:33.908572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:33.908572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:33.908572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:33.908572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:33.908572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:33.908572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:33.908572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:33.908572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:33.908572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:33.908572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:33.908572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:33.908572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:33.908572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:33.908572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:33.908572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:33.908572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:33.908572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:13:33.910337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:13:33.910337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:33.910337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:13:33.910337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 613/1210] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:33.915658 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3f599-258b-46c9-80bb-9e7b06a64a7c None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:34.052302 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:34.057691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3f599-258b-46c9-80bb-9e7b06a64a7c None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:34.102897 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2ee36-0e22-4f1f-a1bc-4439047f257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:34.103505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2ee36-0e22-4f1f-a1bc-4439047f257d None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:34.103859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2ee36-0e22-4f1f-a1bc-4439047f257d None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:34.104149 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2ee36-0e22-4f1f-a1bc-4439047f257d None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:34.104807 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2ee36-0e22-4f1f-a1bc-4439047f257d None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:34.105109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2ee36-0e22-4f1f-a1bc-4439047f257d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:34.106482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2ee36-0e22-4f1f-a1bc-4439047f257d None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:34.119798 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 598/1211] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:34.127066 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec958e39-a367-4cf3-8741-1db97244f104 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:34.246391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec958e39-a367-4cf3-8741-1db97244f104 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:34.249009 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:34.289404 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f2256-2c62-4ec7-8914-ee0703148c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:34.290039 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f2256-2c62-4ec7-8914-ee0703148c1e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:34.290343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f2256-2c62-4ec7-8914-ee0703148c1e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:34.290579 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f2256-2c62-4ec7-8914-ee0703148c1e None admin] PATH_INFO: `/v3/users/placement` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:34.300995 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f2256-2c62-4ec7-8914-ee0703148c1e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:34.303282 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f2256-2c62-4ec7-8914-ee0703148c1e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:34.319190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-c35f2256-2c62-4ec7-8914-ee0703148c1e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:13:34.319190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:34.319190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:34.319190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:34.319190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:34.319190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:34.319190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:34.319190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:34.319190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:34.319190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:34.319190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:34.319190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:34.319190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:13:34.319190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:13:34.319190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:13:34.319190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:34.320530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:34.321512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:34.321512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:34.321512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:13:34.321512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:13:34.321512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:13:34.321512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:13:34.321512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:13:34.321512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:13:34.321512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:13:34.321512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:13:34.321512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 614/1212] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:13:34.327293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2ee36-0e22-4f1f-a1bc-4439047f257d None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:34.439133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2ee36-0e22-4f1f-a1bc-4439047f257d None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:34.475177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58bd73-daf1-4f77-a970-54e54b676645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:34.475805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58bd73-daf1-4f77-a970-54e54b676645 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:34.476079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58bd73-daf1-4f77-a970-54e54b676645 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:34.476350 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58bd73-daf1-4f77-a970-54e54b676645 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:34.477094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58bd73-daf1-4f77-a970-54e54b676645 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:34.477394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58bd73-daf1-4f77-a970-54e54b676645 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:34.478841 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58bd73-daf1-4f77-a970-54e54b676645 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:34.499515 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 599/1213] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Sep 7 13:13:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:34.505574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f2256-2c62-4ec7-8914-ee0703148c1e None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:34.618891 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f2256-2c62-4ec7-8914-ee0703148c1e None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:34.655615 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd148a8-5f09-416b-bc99-737032cfdab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:34.656224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd148a8-5f09-416b-bc99-737032cfdab3 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:34.656552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd148a8-5f09-416b-bc99-737032cfdab3 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:34.656824 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd148a8-5f09-416b-bc99-737032cfdab3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:34.691839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:34.733170 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd148a8-5f09-416b-bc99-737032cfdab3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:34.734622 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd148a8-5f09-416b-bc99-737032cfdab3 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:34.844776 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 615/1214] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:13:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 13:13:34.852100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58bd73-daf1-4f77-a970-54e54b676645 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:34.871665 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:34.969173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58bd73-daf1-4f77-a970-54e54b676645 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:35.007368 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d1f35-cdc3-4956-9093-9d341d15cc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:35.008116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d1f35-cdc3-4956-9093-9d341d15cc8c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:35.008400 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d1f35-cdc3-4956-9093-9d341d15cc8c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:35.008671 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d1f35-cdc3-4956-9093-9d341d15cc8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:35.020546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d1f35-cdc3-4956-9093-9d341d15cc8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:35.022399 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d1f35-cdc3-4956-9093-9d341d15cc8c None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:35.044780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-a43d1f35-cdc3-4956-9093-9d341d15cc8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:35.044780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:35.044780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:35.044780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:35.044780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:35.044780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:35.044780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:35.044780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:35.044780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:35.044780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:35.044780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:35.044780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:35.044780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:13:35.044780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:13:35.044780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:13:35.044780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:13:35.046096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:35.046096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:35.046096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:35.046096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:35.046096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:35.046096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:35.046096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:35.046096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:35.046096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:35.046096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:35.046096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:35.046096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:35.046096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:35.046096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:35.046096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:35.046096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:35.046096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:13:35.047051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:13:35.047051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:35.047051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:13:35.047051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 600/1215] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:35.052573 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd148a8-5f09-416b-bc99-737032cfdab3 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:35.121200 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:35.160961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd148a8-5f09-416b-bc99-737032cfdab3 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:35.197329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d7700-5a2a-4084-81b5-85921994147e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:35.197838 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d7700-5a2a-4084-81b5-85921994147e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:35.198122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d7700-5a2a-4084-81b5-85921994147e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:35.198403 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d7700-5a2a-4084-81b5-85921994147e None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:35.199070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d7700-5a2a-4084-81b5-85921994147e None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:35.199379 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d7700-5a2a-4084-81b5-85921994147e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:35.201156 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d7700-5a2a-4084-81b5-85921994147e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:35.219990 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 616/1216] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:35.226279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d1f35-cdc3-4956-9093-9d341d15cc8c None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:35.315348 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:35.346789 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d1f35-cdc3-4956-9093-9d341d15cc8c None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:35.381041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b7539-65c0-4696-930e-77b285de2541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:35.381671 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b7539-65c0-4696-930e-77b285de2541 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:35.381954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b7539-65c0-4696-930e-77b285de2541 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:35.382217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b7539-65c0-4696-930e-77b285de2541 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:35.389213 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b7539-65c0-4696-930e-77b285de2541 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:35.390748 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b7539-65c0-4696-930e-77b285de2541 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:35.402312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-735b7539-65c0-4696-930e-77b285de2541 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:13:35.402312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:35.402312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:35.402312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:35.402312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:35.402312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:35.402312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:35.402312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:35.402312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:35.402312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:35.402312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:35.402312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:35.402312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:13:35.402312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:13:35.402312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:13:35.404015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:13:35.404015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:35.404015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:35.404015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:35.404015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:35.404015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:35.404015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:35.404015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:35.404015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:35.404015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:35.404015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:35.404015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:35.404015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:35.404015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:35.404015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:35.404015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:35.404015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:35.406272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:13:35.406272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:13:35.406272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:13:35.406272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:13:35.406272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:13:35.406272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:13:35.406272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:13:35.406272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:13:35.406272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 601/1217] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:13:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:35.410581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d7700-5a2a-4084-81b5-85921994147e None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:35.534632 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d7700-5a2a-4084-81b5-85921994147e None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:35.559284 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:35.578218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f263605-5ee4-4154-8f37-8943a3609c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:35.579002 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f263605-5ee4-4154-8f37-8943a3609c20 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:35.579402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f263605-5ee4-4154-8f37-8943a3609c20 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:35.579700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f263605-5ee4-4154-8f37-8943a3609c20 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:35.580395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f263605-5ee4-4154-8f37-8943a3609c20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:35.580715 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f263605-5ee4-4154-8f37-8943a3609c20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:35.582191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f263605-5ee4-4154-8f37-8943a3609c20 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:35.596740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 617/1218] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:13:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:35.604362 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b7539-65c0-4696-930e-77b285de2541 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:35.725711 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b7539-65c0-4696-930e-77b285de2541 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:35.761395 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:35.767588 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9c8a6-a4e8-49f5-8cf3-2eaf901bf4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:35.768199 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9c8a6-a4e8-49f5-8cf3-2eaf901bf4a0 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:35.768476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9c8a6-a4e8-49f5-8cf3-2eaf901bf4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:35.768778 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9c8a6-a4e8-49f5-8cf3-2eaf901bf4a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:35.769470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9c8a6-a4e8-49f5-8cf3-2eaf901bf4a0 None admin] RBAC: Adding query filter params (role.id=743ed2d8e7114975bd9245742a41fe0f, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=bee90bf951f04c2e8e19ceeacb8cfcb2) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:35.769772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9c8a6-a4e8-49f5-8cf3-2eaf901bf4a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:35.771256 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9c8a6-a4e8-49f5-8cf3-2eaf901bf4a0 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:35.776780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 602/1219] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:13:35 2022] GET /identity/v3/role_assignments?user.id=bee90bf951f04c2e8e19ceeacb8cfcb2&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=743ed2d8e7114975bd9245742a41fe0f => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:38.017274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93736db1-8b75-4e23-b7da-f09ef278be06 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:38.017874 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93736db1-8b75-4e23-b7da-f09ef278be06 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:38.017874 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93736db1-8b75-4e23-b7da-f09ef278be06 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:38.019288 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 618/1220] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:13:38.024674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703baec-f310-4a2a-80ef-4b03a8916354 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:38.025131 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703baec-f310-4a2a-80ef-4b03a8916354 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:38.025235 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703baec-f310-4a2a-80ef-4b03a8916354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:38.045507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-f703baec-f310-4a2a-80ef-4b03a8916354 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:13:38.046686 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f703baec-f310-4a2a-80ef-4b03a8916354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:13:38.119454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 603/1221] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:13:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:13:38.415249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93736db1-8b75-4e23-b7da-f09ef278be06 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:38.535331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93736db1-8b75-4e23-b7da-f09ef278be06 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:38.573498 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63186f32-430c-4231-8d58-228a1c69801a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:38.574145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63186f32-430c-4231-8d58-228a1c69801a None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:38.574418 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63186f32-430c-4231-8d58-228a1c69801a None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:38.574693 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63186f32-430c-4231-8d58-228a1c69801a None admin] PATH_INFO: `/v3/roles/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:38.592412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63186f32-430c-4231-8d58-228a1c69801a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:13:38.597387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-63186f32-430c-4231-8d58-228a1c69801a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:13:38.597387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:38.597387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:38.597387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:38.597387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:38.597387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:38.597387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:38.597387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:38.597387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:38.597387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:38.597387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:38.597387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:38.597387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:13:38.597387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:13:38.597387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:13:38.597387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:13:38.603225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:13:38.603225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:13:38.603225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:13:38.603225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:13:38.603225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:38.603225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:38.603225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:38.603225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:38.603225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:38.603225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:38.603225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:38.603225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:38.603225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:38.603225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:38.603225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:38.603225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:38.604387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:38.604387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:38.604387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:38.604387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:38.604387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:13:38.604387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:13:38.604387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:13:38.604387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:13:38.604387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:13:38.604387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:13:38.604387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:13:38.604387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:13:38.604387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 619/1222] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:13:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:38.605315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703baec-f310-4a2a-80ef-4b03a8916354 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:38.710580 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703baec-f310-4a2a-80ef-4b03a8916354 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:38.738547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:38.749327 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca4661-cd9a-4642-bdfa-06a1479cfb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:38.749947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca4661-cd9a-4642-bdfa-06a1479cfb0c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:38.750228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca4661-cd9a-4642-bdfa-06a1479cfb0c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:38.750503 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca4661-cd9a-4642-bdfa-06a1479cfb0c None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:38.751221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca4661-cd9a-4642-bdfa-06a1479cfb0c None admin] RBAC: Adding query filter params (name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:38.751520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca4661-cd9a-4642-bdfa-06a1479cfb0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:38.752889 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca4661-cd9a-4642-bdfa-06a1479cfb0c None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:38.761552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 604/1223] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:13:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:38.768002 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63186f32-430c-4231-8d58-228a1c69801a None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:38.816901 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:38.875722 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63186f32-430c-4231-8d58-228a1c69801a None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:38.912873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c35c25-0b7f-4a87-a6ff-3b3c0f7e9ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:38.913523 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c35c25-0b7f-4a87-a6ff-3b3c0f7e9ccb None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:38.913804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c35c25-0b7f-4a87-a6ff-3b3c0f7e9ccb None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:38.914076 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c35c25-0b7f-4a87-a6ff-3b3c0f7e9ccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:38.926078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c35c25-0b7f-4a87-a6ff-3b3c0f7e9ccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:38.927563 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c35c25-0b7f-4a87-a6ff-3b3c0f7e9ccb None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:38.943953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-66c35c25-0b7f-4a87-a6ff-3b3c0f7e9ccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:38.943953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:38.943953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:38.943953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:38.943953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:38.943953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:38.943953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:38.943953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:38.943953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:38.943953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:38.943953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:38.943953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:38.943953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:13:38.943953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:13:38.943953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:13:38.943953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:13:38.945274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:38.945274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:38.945274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:38.945274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:38.945274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:38.945274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:38.945274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:38.945274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:38.945274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:38.945274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:38.945274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:38.945274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:38.945274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:38.945274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:38.945274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:38.945274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:38.945274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:13:38.946471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:13:38.946471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:38.946471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:13:38.946471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 620/1224] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:38.951940 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca4661-cd9a-4642-bdfa-06a1479cfb0c None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:39.056187 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca4661-cd9a-4642-bdfa-06a1479cfb0c None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:39.096538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfc9b5-25d6-4d24-b910-6de7be176ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:39.097152 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfc9b5-25d6-4d24-b910-6de7be176ecf None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:39.097434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfc9b5-25d6-4d24-b910-6de7be176ecf None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:39.097703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfc9b5-25d6-4d24-b910-6de7be176ecf None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:39.098345 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cfc9b5-25d6-4d24-b910-6de7be176ecf None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:39.098650 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cfc9b5-25d6-4d24-b910-6de7be176ecf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:39.100082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cfc9b5-25d6-4d24-b910-6de7be176ecf None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:39.112845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 605/1225] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:39.118659 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c35c25-0b7f-4a87-a6ff-3b3c0f7e9ccb None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:39.217431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:39.224380 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c35c25-0b7f-4a87-a6ff-3b3c0f7e9ccb None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:39.260498 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635e691-bfd4-4fda-9435-6e9a6c87af4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:39.261162 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635e691-bfd4-4fda-9435-6e9a6c87af4d None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:39.261440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635e691-bfd4-4fda-9435-6e9a6c87af4d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:39.261710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635e691-bfd4-4fda-9435-6e9a6c87af4d None admin] PATH_INFO: `/v3/users/placement` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:39.270922 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635e691-bfd4-4fda-9435-6e9a6c87af4d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:39.272442 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635e691-bfd4-4fda-9435-6e9a6c87af4d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:39.285471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-1635e691-bfd4-4fda-9435-6e9a6c87af4d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:13:39.285471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:39.285471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:39.285471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:39.285471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:39.285471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:39.285471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:39.285471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:39.285471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:39.285471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:39.285471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:39.285471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:39.285471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:13:39.285471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:13:39.285471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:13:39.285471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:39.286709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:39.288065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:39.288065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:39.288065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:13:39.288065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:13:39.288065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:13:39.288065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:13:39.288065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:13:39.288065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:13:39.288065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:13:39.288065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:13:39.288065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 621/1226] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:39.292861 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfc9b5-25d6-4d24-b910-6de7be176ecf None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:39.341492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:39.395435 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfc9b5-25d6-4d24-b910-6de7be176ecf None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:39.429639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d8bc9-4b06-474f-8ba0-f64067d206d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:39.430254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d8bc9-4b06-474f-8ba0-f64067d206d3 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:39.430535 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d8bc9-4b06-474f-8ba0-f64067d206d3 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:39.430802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d8bc9-4b06-474f-8ba0-f64067d206d3 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:39.431574 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d8bc9-4b06-474f-8ba0-f64067d206d3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:39.431874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d8bc9-4b06-474f-8ba0-f64067d206d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:39.433397 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800d8bc9-4b06-474f-8ba0-f64067d206d3 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:39.453525 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 606/1227] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Sep 7 13:13:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:39.459496 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635e691-bfd4-4fda-9435-6e9a6c87af4d None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:39.564250 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635e691-bfd4-4fda-9435-6e9a6c87af4d None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:39.608305 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c91d4-f4e6-4cc8-b810-0bad64dec281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:39.609081 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c91d4-f4e6-4cc8-b810-0bad64dec281 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:39.609528 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c91d4-f4e6-4cc8-b810-0bad64dec281 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:39.609924 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c91d4-f4e6-4cc8-b810-0bad64dec281 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:39.621472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c91d4-f4e6-4cc8-b810-0bad64dec281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:39.622969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c91d4-f4e6-4cc8-b810-0bad64dec281 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:39.637331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-495c91d4-f4e6-4cc8-b810-0bad64dec281 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:39.637331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:39.637331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:39.637331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:39.637331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:39.637331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:39.637331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:39.637331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:39.637331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:39.637331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:39.637331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:39.637331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:39.637331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:13:39.637331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:13:39.637331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:13:39.637331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:13:39.639421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:39.639421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:39.639421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:39.639421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:39.639421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:39.639421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:39.639421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:39.639421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:39.639421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:39.639421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:39.639421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:39.639421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:39.639421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:39.639421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:39.639421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:39.639421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:39.639421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:13:39.640646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:13:39.640646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:39.640646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:13:39.640646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 622/1228] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:39.644977 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d8bc9-4b06-474f-8ba0-f64067d206d3 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:39.755445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d8bc9-4b06-474f-8ba0-f64067d206d3 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:39.757809 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:39.792331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b780ee-8b97-4228-ade0-96179cf1f2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:39.792945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b780ee-8b97-4228-ade0-96179cf1f2cf None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:39.793225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b780ee-8b97-4228-ade0-96179cf1f2cf None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:39.793497 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b780ee-8b97-4228-ade0-96179cf1f2cf None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:39.794154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b780ee-8b97-4228-ade0-96179cf1f2cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:39.794935 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b780ee-8b97-4228-ade0-96179cf1f2cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:39.795854 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b780ee-8b97-4228-ade0-96179cf1f2cf None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:39.807649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 607/1229] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:39.813862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c91d4-f4e6-4cc8-b810-0bad64dec281 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:39.823984 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:39.929201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c91d4-f4e6-4cc8-b810-0bad64dec281 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:39.966931 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f38f68-d1f9-4a35-94ae-2bcd99b597a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:39.979193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f38f68-d1f9-4a35-94ae-2bcd99b597a2 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:39.979193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f38f68-d1f9-4a35-94ae-2bcd99b597a2 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:39.979193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f38f68-d1f9-4a35-94ae-2bcd99b597a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:39.979193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f38f68-d1f9-4a35-94ae-2bcd99b597a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:39.979193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f38f68-d1f9-4a35-94ae-2bcd99b597a2 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:39.988821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-f7f38f68-d1f9-4a35-94ae-2bcd99b597a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:13:39.988821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:39.988821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:39.988821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:39.988821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:39.988821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:39.988821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:39.988821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:39.988821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:39.988821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:39.988821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:39.988821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:39.988821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:13:39.988821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:13:39.988821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:13:39.990051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:13:39.990051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:39.990051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:39.990051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:39.990051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:39.990051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:39.990051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:39.990051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:39.990051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:39.990051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:39.990051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:39.990051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:39.990051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:39.990051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:39.990051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:39.990051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:39.990051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:39.991179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:13:39.991179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:13:39.991179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:13:39.991179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:13:39.991179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:13:39.991179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:13:39.991179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:13:39.991179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:13:39.991179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 623/1230] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:13:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:39.995967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b780ee-8b97-4228-ade0-96179cf1f2cf None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:40.105583 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b780ee-8b97-4228-ade0-96179cf1f2cf None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:40.141157 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dab0ba-a4ac-4e17-a845-cbaaf24a9b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:40.141998 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dab0ba-a4ac-4e17-a845-cbaaf24a9b4a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:40.142224 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dab0ba-a4ac-4e17-a845-cbaaf24a9b4a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:40.142502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dab0ba-a4ac-4e17-a845-cbaaf24a9b4a None admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:40.143302 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dab0ba-a4ac-4e17-a845-cbaaf24a9b4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:40.143610 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dab0ba-a4ac-4e17-a845-cbaaf24a9b4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:40.145056 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dab0ba-a4ac-4e17-a845-cbaaf24a9b4a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:40.157298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 608/1231] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:13:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:40.163613 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f38f68-d1f9-4a35-94ae-2bcd99b597a2 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:40.233472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:40.270440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f38f68-d1f9-4a35-94ae-2bcd99b597a2 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:40.304696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa1cce-1cb3-4399-8c7a-fbee627a049e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:40.305373 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa1cce-1cb3-4399-8c7a-fbee627a049e None admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:40.305646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa1cce-1cb3-4399-8c7a-fbee627a049e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:40.306008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa1cce-1cb3-4399-8c7a-fbee627a049e None admin] PATH_INFO: `/v3/projects/4574cf3ff270485faf60594019f7e05c/users/bee90bf951f04c2e8e19ceeacb8cfcb2/roles/743ed2d8e7114975bd9245742a41fe0f` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:40.323332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa1cce-1cb3-4399-8c7a-fbee627a049e None admin] RBAC: Authorizing `identity:create_grant(project_id=4574cf3ff270485faf60594019f7e05c, user_id=bee90bf951f04c2e8e19ceeacb8cfcb2, role_id=743ed2d8e7114975bd9245742a41fe0f)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:40.325077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa1cce-1cb3-4399-8c7a-fbee627a049e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:40.425173 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 624/1232] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:13:40 2022] PUT /identity/v3/projects/4574cf3ff270485faf60594019f7e05c/users/bee90bf951f04c2e8e19ceeacb8cfcb2/roles/743ed2d8e7114975bd9245742a41fe0f => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:13:42.633571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e9dde-f57b-4ade-b6c1-04702f472cad None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:42.634248 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e9dde-f57b-4ade-b6c1-04702f472cad None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:42.634248 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e9dde-f57b-4ade-b6c1-04702f472cad None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:42.635640 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 609/1233] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:13:42.641884 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334d3bf-b73a-4aa7-a48a-6d10cfd9d249 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:42.642358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334d3bf-b73a-4aa7-a48a-6d10cfd9d249 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:42.642466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334d3bf-b73a-4aa7-a48a-6d10cfd9d249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:42.661277 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-1334d3bf-b73a-4aa7-a48a-6d10cfd9d249 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:13:42.662629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1334d3bf-b73a-4aa7-a48a-6d10cfd9d249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:13:42.735939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 625/1234] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:13:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:13:43.009021 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e9dde-f57b-4ade-b6c1-04702f472cad None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:43.035518 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:43.115452 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e9dde-f57b-4ade-b6c1-04702f472cad None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:43.148476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98f32c-67cc-43fe-8ea3-2384785bbe14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:43.149098 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98f32c-67cc-43fe-8ea3-2384785bbe14 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:43.149375 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98f32c-67cc-43fe-8ea3-2384785bbe14 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:43.149643 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98f32c-67cc-43fe-8ea3-2384785bbe14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:43.160184 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b98f32c-67cc-43fe-8ea3-2384785bbe14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:13:43.165009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-2b98f32c-67cc-43fe-8ea3-2384785bbe14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:13:43.165009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:43.165009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:43.165009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:43.165009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:43.165009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:43.165009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:43.165009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:43.165009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:43.165009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:43.165009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:43.165009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:43.165009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:13:43.165009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:13:43.165009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:13:43.165009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:13:43.166127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:13:43.166127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:13:43.166127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:13:43.166127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:13:43.166127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:43.166127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:43.166127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:43.166127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:43.166127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:43.166127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:43.166127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:43.166127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:43.166127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:43.166127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:43.166127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:43.166127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:43.167068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:43.167068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:43.167068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:43.167068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:43.167068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:13:43.167068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:13:43.167068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:13:43.167068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:13:43.167068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:13:43.167068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:13:43.167068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 13:13:43.167068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:13:43.167068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 610/1235] 10.222.0.62 () {56 vars in 1102 bytes} [Wed Sep 7 13:13:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:43.172928 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334d3bf-b73a-4aa7-a48a-6d10cfd9d249 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:43.222188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:43.275397 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334d3bf-b73a-4aa7-a48a-6d10cfd9d249 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:43.314327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a6ff0-c8a4-40e3-804d-90ecda92402e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:43.315283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a6ff0-c8a4-40e3-804d-90ecda92402e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:43.315965 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a6ff0-c8a4-40e3-804d-90ecda92402e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:43.320883 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a6ff0-c8a4-40e3-804d-90ecda92402e None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:43.320883 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a6ff0-c8a4-40e3-804d-90ecda92402e None admin] RBAC: Adding query filter params (name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:43.320883 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a6ff0-c8a4-40e3-804d-90ecda92402e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:43.320883 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a6ff0-c8a4-40e3-804d-90ecda92402e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:43.327512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 626/1236] 10.222.0.62 () {56 vars in 1103 bytes} [Wed Sep 7 13:13:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:43.334422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98f32c-67cc-43fe-8ea3-2384785bbe14 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:43.422143 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:43.437873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98f32c-67cc-43fe-8ea3-2384785bbe14 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:43.472815 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c14f88-6e0f-44b3-9828-9e2e28811979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:43.473444 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c14f88-6e0f-44b3-9828-9e2e28811979 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:43.473727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c14f88-6e0f-44b3-9828-9e2e28811979 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:43.474002 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c14f88-6e0f-44b3-9828-9e2e28811979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:43.484546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c14f88-6e0f-44b3-9828-9e2e28811979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:43.486042 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c14f88-6e0f-44b3-9828-9e2e28811979 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:43.500865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-30c14f88-6e0f-44b3-9828-9e2e28811979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:43.500865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:43.500865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:43.500865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:43.500865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:43.500865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:43.500865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:43.500865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:43.500865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:43.500865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:43.500865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:43.500865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:43.500865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:13:43.500865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:13:43.500865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:13:43.500865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:13:43.502029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:43.502029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:43.502029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:43.502029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:43.502029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:43.502029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:43.502029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:43.502029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:43.502029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:43.502029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:43.502029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:43.502029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:43.502029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:43.502029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:43.502029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:43.502029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:43.502029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:13:43.503159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:13:43.503159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:43.503159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:13:43.503159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 611/1237] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:43.507922 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a6ff0-c8a4-40e3-804d-90ecda92402e None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:43.625475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a6ff0-c8a4-40e3-804d-90ecda92402e None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:43.634498 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:43.659665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc60b0-d6b6-4f39-832f-7272c2135aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:43.660377 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc60b0-d6b6-4f39-832f-7272c2135aca None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:43.660556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc60b0-d6b6-4f39-832f-7272c2135aca None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:43.660823 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc60b0-d6b6-4f39-832f-7272c2135aca None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:43.661461 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc60b0-d6b6-4f39-832f-7272c2135aca None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:43.661759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc60b0-d6b6-4f39-832f-7272c2135aca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:43.663148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc60b0-d6b6-4f39-832f-7272c2135aca None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:43.674718 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 627/1238] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:43.680717 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c14f88-6e0f-44b3-9828-9e2e28811979 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:43.785631 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c14f88-6e0f-44b3-9828-9e2e28811979 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:43.820367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc7b28-7bc1-4e98-b6fe-f05985593410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:43.820999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc7b28-7bc1-4e98-b6fe-f05985593410 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:43.821283 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc7b28-7bc1-4e98-b6fe-f05985593410 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:43.821560 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc7b28-7bc1-4e98-b6fe-f05985593410 None admin] PATH_INFO: `/v3/users/placement` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:43.824297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:43.833029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc7b28-7bc1-4e98-b6fe-f05985593410 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:43.835785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc7b28-7bc1-4e98-b6fe-f05985593410 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:43.849295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-84cc7b28-7bc1-4e98-b6fe-f05985593410 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:13:43.849295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:43.849295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:43.849295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:43.849295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:43.849295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:43.849295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:43.849295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:43.849295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:43.849295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:43.849295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:43.849295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:43.849295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:13:43.849295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:13:43.849295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:13:43.849295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:43.850576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:43.851790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:43.851790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:43.851790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:13:43.851790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:13:43.851790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:13:43.851790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:13:43.851790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:13:43.851790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:13:43.851790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:13:43.851790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:13:43.851790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 612/1239] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:43.856453 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc60b0-d6b6-4f39-832f-7272c2135aca None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:43.960389 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc60b0-d6b6-4f39-832f-7272c2135aca None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:43.996083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d88f02-3f7e-4fc2-9f9c-c307cc8634df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:43.996809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d88f02-3f7e-4fc2-9f9c-c307cc8634df None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:43.997810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d88f02-3f7e-4fc2-9f9c-c307cc8634df None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:43.998228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d88f02-3f7e-4fc2-9f9c-c307cc8634df None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:43.999246 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d88f02-3f7e-4fc2-9f9c-c307cc8634df None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:43.999633 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d88f02-3f7e-4fc2-9f9c-c307cc8634df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:44.001170 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d88f02-3f7e-4fc2-9f9c-c307cc8634df None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:44.019788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 628/1240] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Sep 7 13:13:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:44.025664 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc7b28-7bc1-4e98-b6fe-f05985593410 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:44.129194 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc7b28-7bc1-4e98-b6fe-f05985593410 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:44.163123 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea6a03-abce-44d8-b63c-f48ddc226f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:44.163734 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea6a03-abce-44d8-b63c-f48ddc226f3b None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:44.164013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea6a03-abce-44d8-b63c-f48ddc226f3b None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:44.164287 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea6a03-abce-44d8-b63c-f48ddc226f3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:44.226739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:44.234386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea6a03-abce-44d8-b63c-f48ddc226f3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:44.235902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea6a03-abce-44d8-b63c-f48ddc226f3b None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:44.338345 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 613/1241] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:13:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:13:44.344294 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d88f02-3f7e-4fc2-9f9c-c307cc8634df None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:44.373455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:44.445880 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d88f02-3f7e-4fc2-9f9c-c307cc8634df None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:44.482742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996c473-49dd-4041-b020-1e1e21a69bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:44.482742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996c473-49dd-4041-b020-1e1e21a69bed None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:44.482742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996c473-49dd-4041-b020-1e1e21a69bed None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:44.482742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996c473-49dd-4041-b020-1e1e21a69bed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:44.495308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996c473-49dd-4041-b020-1e1e21a69bed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:44.496764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996c473-49dd-4041-b020-1e1e21a69bed None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:44.512900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-3996c473-49dd-4041-b020-1e1e21a69bed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:44.512900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:44.512900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:44.512900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:44.512900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:44.512900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:44.512900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:44.512900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:44.512900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:44.512900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:44.512900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:44.512900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:44.512900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:13:44.512900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:13:44.512900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:13:44.512900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:13:44.514089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:44.514089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:44.514089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:44.514089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:44.514089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:44.514089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:44.514089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:44.514089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:44.514089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:44.514089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:44.514089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:44.514089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:44.514089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:44.514089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:44.514089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:44.514089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:44.514089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:13:44.515083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:13:44.515083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:44.515083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:13:44.516417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 629/1242] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:44.521788 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea6a03-abce-44d8-b63c-f48ddc226f3b None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:44.622021 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:44.628693 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea6a03-abce-44d8-b63c-f48ddc226f3b None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:44.664410 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c25f62-1e81-4757-99cc-73323bea4ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:44.665003 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c25f62-1e81-4757-99cc-73323bea4ff8 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:44.665283 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c25f62-1e81-4757-99cc-73323bea4ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:44.665553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c25f62-1e81-4757-99cc-73323bea4ff8 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:44.670571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c25f62-1e81-4757-99cc-73323bea4ff8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:44.670571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c25f62-1e81-4757-99cc-73323bea4ff8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:44.670571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c25f62-1e81-4757-99cc-73323bea4ff8 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:44.679710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 614/1243] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:44.686333 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996c473-49dd-4041-b020-1e1e21a69bed None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:44.769863 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:44.791626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996c473-49dd-4041-b020-1e1e21a69bed None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:44.827341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324ff55-58e8-43cd-8e39-8e5d2fde21e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:44.827982 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324ff55-58e8-43cd-8e39-8e5d2fde21e7 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:44.828261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324ff55-58e8-43cd-8e39-8e5d2fde21e7 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:44.828534 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324ff55-58e8-43cd-8e39-8e5d2fde21e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:44.835482 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324ff55-58e8-43cd-8e39-8e5d2fde21e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:44.836905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324ff55-58e8-43cd-8e39-8e5d2fde21e7 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:44.848866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-f324ff55-58e8-43cd-8e39-8e5d2fde21e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:13:44.848866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:44.848866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:44.848866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:44.848866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:44.848866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:44.848866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:44.848866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:44.848866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:44.848866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:44.848866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:44.848866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:44.848866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:13:44.848866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:13:44.848866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:13:44.849994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:13:44.849994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:44.849994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:44.849994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:44.849994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:44.849994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:44.849994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:44.849994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:44.849994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:44.849994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:44.849994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:44.849994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:44.849994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:44.849994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:44.849994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:44.849994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:44.849994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:44.851033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:13:44.851033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:13:44.851033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:13:44.851033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:13:44.851033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:13:44.851033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:13:44.851033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:13:44.851033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:13:44.851033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 630/1244] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:13:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:44.855896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c25f62-1e81-4757-99cc-73323bea4ff8 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:44.961156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c25f62-1e81-4757-99cc-73323bea4ff8 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:44.997349 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9387a0-2fc1-4ba4-bc51-4d0070952433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:44.998041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9387a0-2fc1-4ba4-bc51-4d0070952433 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:44.998323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9387a0-2fc1-4ba4-bc51-4d0070952433 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:44.998593 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9387a0-2fc1-4ba4-bc51-4d0070952433 None admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:44.999328 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9387a0-2fc1-4ba4-bc51-4d0070952433 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:44.999741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9387a0-2fc1-4ba4-bc51-4d0070952433 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:45.001063 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9387a0-2fc1-4ba4-bc51-4d0070952433 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:45.014086 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 615/1245] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:13:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:45.020375 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324ff55-58e8-43cd-8e39-8e5d2fde21e7 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:45.120890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324ff55-58e8-43cd-8e39-8e5d2fde21e7 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:45.155584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2584c0-fce2-4536-b11c-59393f82571d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:45.156191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2584c0-fce2-4536-b11c-59393f82571d None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:45.156473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2584c0-fce2-4536-b11c-59393f82571d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:45.156740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2584c0-fce2-4536-b11c-59393f82571d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:45.157584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2584c0-fce2-4536-b11c-59393f82571d None admin] RBAC: Adding query filter params (role.id=743ed2d8e7114975bd9245742a41fe0f, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=bee90bf951f04c2e8e19ceeacb8cfcb2) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:45.157892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2584c0-fce2-4536-b11c-59393f82571d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:45.159360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2584c0-fce2-4536-b11c-59393f82571d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:45.165286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 631/1246] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:13:45 2022] GET /identity/v3/role_assignments?user.id=bee90bf951f04c2e8e19ceeacb8cfcb2&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=743ed2d8e7114975bd9245742a41fe0f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:47.396457 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c9b37-043c-46f6-891e-3b26d9401c9b None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:47.396987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c9b37-043c-46f6-891e-3b26d9401c9b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:47.396987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c9b37-043c-46f6-891e-3b26d9401c9b None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:47.398339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 616/1247] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:13:47.404601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdaa19e-416a-4c8e-a785-58a2d9e6e6f8 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:47.407036 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdaa19e-416a-4c8e-a785-58a2d9e6e6f8 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:47.407036 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdaa19e-416a-4c8e-a785-58a2d9e6e6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:47.411793 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:47.424379 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-7fdaa19e-416a-4c8e-a785-58a2d9e6e6f8 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:13:47.425564 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7fdaa19e-416a-4c8e-a785-58a2d9e6e6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:13:47.494293 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 632/1248] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:13:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 13:13:47.769050 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c9b37-043c-46f6-891e-3b26d9401c9b None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:47.787581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:47.874015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c9b37-043c-46f6-891e-3b26d9401c9b None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:47.911715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b38fda-2025-4a64-9acb-719b81f1530c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:47.912345 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b38fda-2025-4a64-9acb-719b81f1530c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:47.912621 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b38fda-2025-4a64-9acb-719b81f1530c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:47.912892 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b38fda-2025-4a64-9acb-719b81f1530c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:47.924243 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b38fda-2025-4a64-9acb-719b81f1530c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:13:47.929057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-f2b38fda-2025-4a64-9acb-719b81f1530c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:13:47.929057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:47.929057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:47.929057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:47.929057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:47.929057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:47.929057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:47.929057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:47.929057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:47.929057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:47.929057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:47.929057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:47.929057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:13:47.929057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:13:47.929057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:13:47.929057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:13:47.930197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:13:47.930197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:13:47.930197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:13:47.930197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:13:47.930197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:47.930197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:47.930197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:47.930197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:47.930197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:47.930197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:47.930197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:47.930197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:47.930197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:47.930197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:47.930197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:47.930197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:47.931573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:47.931573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:47.931573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:47.931573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:47.931573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:13:47.931573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:13:47.931573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:13:47.931573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:13:47.931573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:13:47.931573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:13:47.931573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:13:47.931573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:13:47.931573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 617/1249] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:13:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:47.937023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdaa19e-416a-4c8e-a785-58a2d9e6e6f8 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:48.039425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdaa19e-416a-4c8e-a785-58a2d9e6e6f8 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:48.069657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:48.075438 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23059b6d-6a91-4314-8f40-5886c9dd776d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:48.076052 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23059b6d-6a91-4314-8f40-5886c9dd776d None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:48.076324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23059b6d-6a91-4314-8f40-5886c9dd776d None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:48.076587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23059b6d-6a91-4314-8f40-5886c9dd776d None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:48.077480 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23059b6d-6a91-4314-8f40-5886c9dd776d None admin] RBAC: Adding query filter params (name=admin) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:48.077781 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23059b6d-6a91-4314-8f40-5886c9dd776d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:48.079224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23059b6d-6a91-4314-8f40-5886c9dd776d None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:48.088236 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 633/1250] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:13:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:48.094722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b38fda-2025-4a64-9acb-719b81f1530c None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:48.180706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:48.204599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b38fda-2025-4a64-9acb-719b81f1530c None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:48.238941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a515b0-6528-49d4-bf2b-378bbbef0257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:48.239601 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a515b0-6528-49d4-bf2b-378bbbef0257 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:48.239881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a515b0-6528-49d4-bf2b-378bbbef0257 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:48.240151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a515b0-6528-49d4-bf2b-378bbbef0257 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:48.250676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a515b0-6528-49d4-bf2b-378bbbef0257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:48.252150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a515b0-6528-49d4-bf2b-378bbbef0257 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:48.266250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-90a515b0-6528-49d4-bf2b-378bbbef0257 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:48.266250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:48.266250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:48.266250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:48.266250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:48.266250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:48.266250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:48.266250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:48.266250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:48.266250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:48.266250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:48.266250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:48.266250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:13:48.266250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:13:48.266250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:13:48.266250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:13:48.267490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:48.267490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:48.267490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:48.267490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:48.267490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:48.267490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:48.267490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:48.267490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:48.267490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:48.267490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:48.267490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:48.267490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:48.267490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:48.267490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:48.267490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:48.267490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:48.267490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:13:48.273972 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:13:48.273972 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:48.273972 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:13:48.273972 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 618/1251] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:48.274273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23059b6d-6a91-4314-8f40-5886c9dd776d None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:48.380076 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23059b6d-6a91-4314-8f40-5886c9dd776d None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:48.414136 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb3333-8aa0-49a4-bf47-5a9151a19012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:48.414834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb3333-8aa0-49a4-bf47-5a9151a19012 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:48.415034 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb3333-8aa0-49a4-bf47-5a9151a19012 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:48.415344 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb3333-8aa0-49a4-bf47-5a9151a19012 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:48.415985 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb3333-8aa0-49a4-bf47-5a9151a19012 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:48.416283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb3333-8aa0-49a4-bf47-5a9151a19012 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:48.417639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb3333-8aa0-49a4-bf47-5a9151a19012 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:48.434443 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 634/1252] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:48.440565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a515b0-6528-49d4-bf2b-378bbbef0257 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:48.563401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a515b0-6528-49d4-bf2b-378bbbef0257 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:48.598608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:48.606889 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed784e-1da2-4f26-afe3-5e4510b7d47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:48.607567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed784e-1da2-4f26-afe3-5e4510b7d47a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:48.607851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed784e-1da2-4f26-afe3-5e4510b7d47a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:48.608116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed784e-1da2-4f26-afe3-5e4510b7d47a None admin] PATH_INFO: `/v3/users/placement` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:48.616468 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed784e-1da2-4f26-afe3-5e4510b7d47a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:48.617919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed784e-1da2-4f26-afe3-5e4510b7d47a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:48.629280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-54ed784e-1da2-4f26-afe3-5e4510b7d47a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:13:48.629280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:48.629280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:48.629280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:48.629280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:48.629280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:48.629280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:48.629280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:48.629280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:48.629280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:48.629280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:48.629280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:48.629280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:13:48.629280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:13:48.629280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:13:48.629280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:48.630486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:48.631587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:48.631587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:48.631587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:13:48.631587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:13:48.631587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:13:48.631587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:13:48.631587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:13:48.631587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:13:48.631587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:13:48.631587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:13:48.631587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 619/1253] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:48.636329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb3333-8aa0-49a4-bf47-5a9151a19012 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:48.702622 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:48.753192 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb3333-8aa0-49a4-bf47-5a9151a19012 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:48.791988 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26615bfc-2075-4163-b552-e2d051579181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:48.792602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26615bfc-2075-4163-b552-e2d051579181 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:48.792873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26615bfc-2075-4163-b552-e2d051579181 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:48.793137 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26615bfc-2075-4163-b552-e2d051579181 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:48.793926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26615bfc-2075-4163-b552-e2d051579181 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:48.794387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26615bfc-2075-4163-b552-e2d051579181 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:48.796630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26615bfc-2075-4163-b552-e2d051579181 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:48.817362 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 635/1254] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Sep 7 13:13:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:48.824101 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed784e-1da2-4f26-afe3-5e4510b7d47a None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:48.932722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed784e-1da2-4f26-afe3-5e4510b7d47a None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:48.974250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b467be8-23fc-428d-af50-dcda9534b6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:48.974864 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b467be8-23fc-428d-af50-dcda9534b6df None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:48.975239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b467be8-23fc-428d-af50-dcda9534b6df None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:48.975510 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b467be8-23fc-428d-af50-dcda9534b6df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:49.036214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:49.049485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b467be8-23fc-428d-af50-dcda9534b6df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:49.050959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b467be8-23fc-428d-af50-dcda9534b6df None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:49.166453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 620/1255] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:13:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:13:49.172420 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26615bfc-2075-4163-b552-e2d051579181 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:49.299012 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26615bfc-2075-4163-b552-e2d051579181 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:49.312701 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:49.334423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ab125-3e2e-45eb-a11d-49be28d33bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:49.335082 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ab125-3e2e-45eb-a11d-49be28d33bc8 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:49.335360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ab125-3e2e-45eb-a11d-49be28d33bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:49.335633 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ab125-3e2e-45eb-a11d-49be28d33bc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:49.346174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ab125-3e2e-45eb-a11d-49be28d33bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:49.347735 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ab125-3e2e-45eb-a11d-49be28d33bc8 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:49.362413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-a39ab125-3e2e-45eb-a11d-49be28d33bc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:49.362413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:49.362413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:49.362413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:49.362413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:49.362413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:49.362413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:49.362413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:49.362413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:49.362413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:49.362413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:49.362413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:49.362413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:13:49.362413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:13:49.362413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:13:49.362413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:13:49.370799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:49.370799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:49.370799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:49.370799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:49.370799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:49.370799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:49.370799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:49.370799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:49.370799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:49.370799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:49.370799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:49.370799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:49.370799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:49.370799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:49.370799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:49.370799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:49.370799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:13:49.372295 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:13:49.372295 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:49.372295 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:13:49.372295 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 636/1256] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:49.372611 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b467be8-23fc-428d-af50-dcda9534b6df None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:49.457608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:49.470317 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b467be8-23fc-428d-af50-dcda9534b6df None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:49.506175 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d803ba2-7c40-4c3c-9303-b6eae3469002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:49.506796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d803ba2-7c40-4c3c-9303-b6eae3469002 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:49.507103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d803ba2-7c40-4c3c-9303-b6eae3469002 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:49.507415 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d803ba2-7c40-4c3c-9303-b6eae3469002 None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:49.508062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d803ba2-7c40-4c3c-9303-b6eae3469002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:49.508363 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d803ba2-7c40-4c3c-9303-b6eae3469002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:49.509745 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d803ba2-7c40-4c3c-9303-b6eae3469002 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:49.522155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 621/1257] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:49.528134 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ab125-3e2e-45eb-a11d-49be28d33bc8 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:49.644131 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ab125-3e2e-45eb-a11d-49be28d33bc8 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:49.685757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf503e5b-9ea0-479f-987a-4e1b17317cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:49.686470 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf503e5b-9ea0-479f-987a-4e1b17317cb1 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:49.686928 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf503e5b-9ea0-479f-987a-4e1b17317cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:49.687276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf503e5b-9ea0-479f-987a-4e1b17317cb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:49.696670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf503e5b-9ea0-479f-987a-4e1b17317cb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:49.696670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf503e5b-9ea0-479f-987a-4e1b17317cb1 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:49.708437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-cf503e5b-9ea0-479f-987a-4e1b17317cb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:13:49.708437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:49.708437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:49.708437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:49.708437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:49.708437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:49.708437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:49.708437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:49.708437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:49.708437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:49.708437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:49.708437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:49.708437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:13:49.708437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:13:49.708437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:13:49.709510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:13:49.709510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:49.709510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:49.709510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:49.709510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:49.709510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:49.709510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:49.709510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:49.709510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:49.709510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:49.709510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:49.709510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:49.709510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:49.709510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:49.709510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:49.709510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:49.709510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:49.710537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:13:49.710537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:13:49.710537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:13:49.710537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:13:49.710537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:13:49.710537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:13:49.710537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:13:49.710537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:13:49.710537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 637/1258] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:13:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:49.716078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d803ba2-7c40-4c3c-9303-b6eae3469002 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:49.816579 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d803ba2-7c40-4c3c-9303-b6eae3469002 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:49.850298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624e793-fddf-4245-9efa-94cd2f9fd73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:49.850904 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624e793-fddf-4245-9efa-94cd2f9fd73a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:49.851207 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624e793-fddf-4245-9efa-94cd2f9fd73a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:49.851477 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624e793-fddf-4245-9efa-94cd2f9fd73a None admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:49.852159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624e793-fddf-4245-9efa-94cd2f9fd73a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:49.852466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624e793-fddf-4245-9efa-94cd2f9fd73a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:49.853867 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624e793-fddf-4245-9efa-94cd2f9fd73a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:49.866351 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 622/1259] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:13:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:49.872893 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf503e5b-9ea0-479f-987a-4e1b17317cb1 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:49.900449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:49.978057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf503e5b-9ea0-479f-987a-4e1b17317cb1 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:50.014502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8620b-302f-41ed-a4bc-70821a6ff39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:50.015137 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8620b-302f-41ed-a4bc-70821a6ff39c None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:50.015419 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8620b-302f-41ed-a4bc-70821a6ff39c None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:50.015690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8620b-302f-41ed-a4bc-70821a6ff39c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:50.016441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8620b-302f-41ed-a4bc-70821a6ff39c None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=bee90bf951f04c2e8e19ceeacb8cfcb2) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:50.016841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8620b-302f-41ed-a4bc-70821a6ff39c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:50.018157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8620b-302f-41ed-a4bc-70821a6ff39c None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:50.023555 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 638/1260] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:13:49 2022] GET /identity/v3/role_assignments?user.id=bee90bf951f04c2e8e19ceeacb8cfcb2&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:52.197289 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7f9d8-5001-4768-925e-c4a7afd8f183 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:52.197874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7f9d8-5001-4768-925e-c4a7afd8f183 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:52.197874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7f9d8-5001-4768-925e-c4a7afd8f183 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:52.199669 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 623/1261] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:13:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:13:52.205929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd4435-0d21-4c41-85a5-27049b74b1c9 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:52.206370 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd4435-0d21-4c41-85a5-27049b74b1c9 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:52.206486 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd4435-0d21-4c41-85a5-27049b74b1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:52.225208 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-addd4435-0d21-4c41-85a5-27049b74b1c9 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:13:52.226575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-addd4435-0d21-4c41-85a5-27049b74b1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:13:52.296403 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 639/1262] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:13:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:13:52.568721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7f9d8-5001-4768-925e-c4a7afd8f183 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:52.581013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:52.677881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7f9d8-5001-4768-925e-c4a7afd8f183 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:52.713255 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e861d4-a3bf-473d-a2a4-c9e623ff1d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:52.713883 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e861d4-a3bf-473d-a2a4-c9e623ff1d4c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:52.714157 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e861d4-a3bf-473d-a2a4-c9e623ff1d4c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:52.714423 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e861d4-a3bf-473d-a2a4-c9e623ff1d4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:52.726162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e861d4-a3bf-473d-a2a4-c9e623ff1d4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:13:52.732678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-d4e861d4-a3bf-473d-a2a4-c9e623ff1d4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:13:52.732678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:52.732678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:52.732678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:52.732678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:52.732678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:52.732678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:52.732678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:52.732678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:52.732678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:52.732678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:52.732678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:52.732678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:13:52.732678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:13:52.732678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:13:52.732678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:13:52.733814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:13:52.733814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:13:52.733814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:13:52.733814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:13:52.733814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:52.733814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:52.733814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:52.733814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:52.733814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:52.733814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:52.733814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:52.733814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:52.733814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:52.733814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:52.733814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:52.733814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:52.734679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:52.734679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:52.734679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:52.734679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:52.734679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:13:52.734679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:13:52.734679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:13:52.734679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:13:52.734679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:13:52.734679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:13:52.734679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:13:52.734679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:13:52.734679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 624/1263] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:13:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:52.738804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd4435-0d21-4c41-85a5-27049b74b1c9 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:52.758847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:52.842865 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd4435-0d21-4c41-85a5-27049b74b1c9 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:52.879804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438e8dd-b1a1-4fba-bede-771ebfdab6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:52.880457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438e8dd-b1a1-4fba-bede-771ebfdab6e1 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:52.880737 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438e8dd-b1a1-4fba-bede-771ebfdab6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:52.880999 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1438e8dd-b1a1-4fba-bede-771ebfdab6e1 None admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:52.881695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438e8dd-b1a1-4fba-bede-771ebfdab6e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:52.881994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438e8dd-b1a1-4fba-bede-771ebfdab6e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:52.883367 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1438e8dd-b1a1-4fba-bede-771ebfdab6e1 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:52.892744 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 640/1264] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:13:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:52.898641 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e861d4-a3bf-473d-a2a4-c9e623ff1d4c None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:52.967653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:53.001697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e861d4-a3bf-473d-a2a4-c9e623ff1d4c None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:53.037122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97670a1-89bb-4673-b2a5-bb3cab2a0075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:53.037750 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97670a1-89bb-4673-b2a5-bb3cab2a0075 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:53.038027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97670a1-89bb-4673-b2a5-bb3cab2a0075 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:53.038293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97670a1-89bb-4673-b2a5-bb3cab2a0075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:53.048776 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97670a1-89bb-4673-b2a5-bb3cab2a0075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:53.050220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97670a1-89bb-4673-b2a5-bb3cab2a0075 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:53.065485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-b97670a1-89bb-4673-b2a5-bb3cab2a0075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:53.065485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:53.065485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:53.065485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:53.065485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:53.065485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:53.065485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:53.065485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:53.065485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:53.065485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:53.065485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:53.065485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:53.065485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:13:53.065485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:13:53.065485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:13:53.065485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:13:53.066653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:53.066653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:53.066653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:53.066653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:53.066653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:53.066653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:53.066653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:53.066653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:53.066653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:53.066653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:53.066653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:53.066653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:53.066653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:53.066653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:53.066653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:53.066653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:53.066653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:13:53.067803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:13:53.067803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:53.067803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:13:53.067803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 625/1265] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:53.072543 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438e8dd-b1a1-4fba-bede-771ebfdab6e1 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:53.157727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:53.176645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1438e8dd-b1a1-4fba-bede-771ebfdab6e1 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:53.212457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157f372-633f-41c2-8953-3b74f79db034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:53.213084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157f372-633f-41c2-8953-3b74f79db034 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:53.213331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157f372-633f-41c2-8953-3b74f79db034 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:53.213604 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157f372-633f-41c2-8953-3b74f79db034 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:53.214302 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157f372-633f-41c2-8953-3b74f79db034 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:53.214598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157f372-633f-41c2-8953-3b74f79db034 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:53.216749 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157f372-633f-41c2-8953-3b74f79db034 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:53.227853 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 641/1266] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:53.234386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97670a1-89bb-4673-b2a5-bb3cab2a0075 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:53.336769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97670a1-89bb-4673-b2a5-bb3cab2a0075 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:53.357427 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:53.370902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fea00-4c7c-481f-bcc1-4ef746355332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:53.372146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fea00-4c7c-481f-bcc1-4ef746355332 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:53.372435 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fea00-4c7c-481f-bcc1-4ef746355332 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:53.372705 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fea00-4c7c-481f-bcc1-4ef746355332 None admin] PATH_INFO: `/v3/users/placement` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:53.380367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0fea00-4c7c-481f-bcc1-4ef746355332 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:53.381835 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0fea00-4c7c-481f-bcc1-4ef746355332 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:53.392885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-0c0fea00-4c7c-481f-bcc1-4ef746355332 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:13:53.392885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:53.392885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:53.392885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:53.392885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:53.392885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:53.392885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:53.392885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:53.392885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:53.392885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:53.392885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:53.392885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:53.392885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:13:53.392885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:13:53.392885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:13:53.392885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:53.395329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:53.396691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:53.396691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:53.396691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:13:53.396691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:13:53.396691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:13:53.396691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:13:53.396691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:13:53.396691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:13:53.396691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:13:53.396691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:13:53.396691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 626/1267] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:53.404199 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157f372-633f-41c2-8953-3b74f79db034 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:53.508083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157f372-633f-41c2-8953-3b74f79db034 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:53.542542 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704ec48-b96b-4ef0-a97f-c564600b11a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:53.543214 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704ec48-b96b-4ef0-a97f-c564600b11a9 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:53.544087 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704ec48-b96b-4ef0-a97f-c564600b11a9 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:53.544363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704ec48-b96b-4ef0-a97f-c564600b11a9 None admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:53.545099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704ec48-b96b-4ef0-a97f-c564600b11a9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:53.545413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704ec48-b96b-4ef0-a97f-c564600b11a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:53.546841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704ec48-b96b-4ef0-a97f-c564600b11a9 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:53.556612 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:53.565128 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 642/1268] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Sep 7 13:13:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:53.571095 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fea00-4c7c-481f-bcc1-4ef746355332 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:53.676670 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fea00-4c7c-481f-bcc1-4ef746355332 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:53.711281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2db51-d3bb-4fe5-a415-5b95a3db37b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:53.711948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2db51-d3bb-4fe5-a415-5b95a3db37b6 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:53.712233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2db51-d3bb-4fe5-a415-5b95a3db37b6 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:53.712510 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2db51-d3bb-4fe5-a415-5b95a3db37b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:53.724946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2db51-d3bb-4fe5-a415-5b95a3db37b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:53.726471 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2db51-d3bb-4fe5-a415-5b95a3db37b6 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:53.742239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-35b2db51-d3bb-4fe5-a415-5b95a3db37b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:53.742239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:53.742239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:53.742239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:53.742239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:53.742239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:53.742239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:53.742239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:53.742239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:53.742239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:53.742239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:53.742239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:53.742239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:13:53.742239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:13:53.742239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:13:53.742239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:13:53.743507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:53.743507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:53.743507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:53.743507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:53.743507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:53.743507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:53.743507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:53.743507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:53.743507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:53.743507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:53.743507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:53.743507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:53.743507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:53.743507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:53.743507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:53.743507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:53.743507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:13:53.744451 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:13:53.744451 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:53.744451 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:13:53.744451 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 627/1269] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:53.749432 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704ec48-b96b-4ef0-a97f-c564600b11a9 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:53.856240 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704ec48-b96b-4ef0-a97f-c564600b11a9 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:53.893309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ba33c-ef7e-4e7b-a056-14c0cedc428e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:53.893975 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ba33c-ef7e-4e7b-a056-14c0cedc428e None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:53.894271 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ba33c-ef7e-4e7b-a056-14c0cedc428e None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:53.894541 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ba33c-ef7e-4e7b-a056-14c0cedc428e None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:53.895217 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ba33c-ef7e-4e7b-a056-14c0cedc428e None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:53.895571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ba33c-ef7e-4e7b-a056-14c0cedc428e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:53.896972 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ba33c-ef7e-4e7b-a056-14c0cedc428e None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:53.909656 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 643/1270] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:53.920432 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2db51-d3bb-4fe5-a415-5b95a3db37b6 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:53.947351 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:54.017358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2db51-d3bb-4fe5-a415-5b95a3db37b6 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:54.052288 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6d000-622d-4dfc-8021-975a56ed8695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:54.052923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6d000-622d-4dfc-8021-975a56ed8695 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:54.053200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6d000-622d-4dfc-8021-975a56ed8695 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:54.053468 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6d000-622d-4dfc-8021-975a56ed8695 None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:54.060581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a6d000-622d-4dfc-8021-975a56ed8695 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:54.062165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a6d000-622d-4dfc-8021-975a56ed8695 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:54.073654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-54a6d000-622d-4dfc-8021-975a56ed8695 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:13:54.073654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:54.073654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:54.073654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:54.073654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:54.073654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:54.073654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:54.073654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:54.073654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:54.073654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:54.073654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:54.073654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:54.073654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:13:54.073654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:13:54.073654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:13:54.076459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:13:54.076459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:54.076459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:54.076459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:54.076459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:54.076459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:54.076459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:54.076459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:54.076459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:54.076459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:54.076459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:54.076459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:54.076459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:54.076459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:54.076459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:54.076459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:54.076459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:54.077590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:13:54.077590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:13:54.077590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:13:54.077590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:13:54.077590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:13:54.077590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:13:54.077590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:13:54.077590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:13:54.077590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 628/1271] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:13:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:54.081960 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ba33c-ef7e-4e7b-a056-14c0cedc428e None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:54.142667 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:54.188156 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ba33c-ef7e-4e7b-a056-14c0cedc428e None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:54.224615 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b89cc-0fdc-44db-a4ff-8fe0d41ede71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:54.225255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b89cc-0fdc-44db-a4ff-8fe0d41ede71 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:54.225492 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b89cc-0fdc-44db-a4ff-8fe0d41ede71 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:54.225763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b89cc-0fdc-44db-a4ff-8fe0d41ede71 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:54.226449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b89cc-0fdc-44db-a4ff-8fe0d41ede71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:54.226752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b89cc-0fdc-44db-a4ff-8fe0d41ede71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:54.228210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b89cc-0fdc-44db-a4ff-8fe0d41ede71 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:54.240208 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 644/1272] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:13:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:54.246328 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6d000-622d-4dfc-8021-975a56ed8695 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:54.342896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:54.351749 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6d000-622d-4dfc-8021-975a56ed8695 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:54.385769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07862d83-44ed-4642-a57d-fa3ca5992eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:54.386427 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07862d83-44ed-4642-a57d-fa3ca5992eff None admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:54.386708 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07862d83-44ed-4642-a57d-fa3ca5992eff None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:54.386998 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07862d83-44ed-4642-a57d-fa3ca5992eff None admin] PATH_INFO: `/v3/projects/4574cf3ff270485faf60594019f7e05c/users/bee90bf951f04c2e8e19ceeacb8cfcb2/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:54.389589 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07862d83-44ed-4642-a57d-fa3ca5992eff None admin] RBAC: Authorizing `identity:create_grant(project_id=4574cf3ff270485faf60594019f7e05c, user_id=bee90bf951f04c2e8e19ceeacb8cfcb2, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:54.391053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07862d83-44ed-4642-a57d-fa3ca5992eff None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:54.516101 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 629/1273] 10.222.0.62 () {58 vars in 1439 bytes} [Wed Sep 7 13:13:54 2022] PUT /identity/v3/projects/4574cf3ff270485faf60594019f7e05c/users/bee90bf951f04c2e8e19ceeacb8cfcb2/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:13:56.699239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9caf6-fb5d-45a5-98b3-24cea512c66d None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:56.700590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9caf6-fb5d-45a5-98b3-24cea512c66d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:56.701341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9caf6-fb5d-45a5-98b3-24cea512c66d None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:56.702790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 645/1274] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:13:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:13:56.708821 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525fb89-6a96-407d-9db3-737af51a6bc3 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:56.709437 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525fb89-6a96-407d-9db3-737af51a6bc3 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:56.709437 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525fb89-6a96-407d-9db3-737af51a6bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:56.728357 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-7525fb89-6a96-407d-9db3-737af51a6bc3 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:13:56.729555 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-7525fb89-6a96-407d-9db3-737af51a6bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:13:56.799212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 630/1275] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:13:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:13:57.087012 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9caf6-fb5d-45a5-98b3-24cea512c66d None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:57.201379 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9caf6-fb5d-45a5-98b3-24cea512c66d None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:57.221122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:57.239544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e88e4-25e1-4184-8e27-d68834778db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:57.240173 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e88e4-25e1-4184-8e27-d68834778db8 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:57.242416 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e88e4-25e1-4184-8e27-d68834778db8 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:57.242416 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e88e4-25e1-4184-8e27-d68834778db8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:57.252239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e88e4-25e1-4184-8e27-d68834778db8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 13:13:57.257172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-6e9e88e4-25e1-4184-8e27-d68834778db8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:13:57.257172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:57.257172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:57.257172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:57.257172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:57.257172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:57.257172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:57.257172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:57.257172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:57.257172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:57.257172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:57.257172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:57.257172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 13:13:57.257172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 13:13:57.257172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 13:13:57.257172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 13:13:57.258920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 13:13:57.258920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 13:13:57.258920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 13:13:57.258920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 13:13:57.258920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:57.258920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:57.258920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:57.258920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:57.258920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:57.258920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:57.258920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:57.258920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:57.258920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:57.258920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:57.258920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:57.258920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:57.260119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:57.260119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:57.260119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:57.260119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:57.260119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 13:13:57.260119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 13:13:57.260119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 13:13:57.260119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 13:13:57.260119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 13:13:57.260119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 13:13:57.260119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 13:13:57.260119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:13:57.260119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 646/1276] 10.222.0.62 () {56 vars in 1096 bytes} [Wed Sep 7 13:13:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:57.265075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525fb89-6a96-407d-9db3-737af51a6bc3 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:57.349620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:57.367838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525fb89-6a96-407d-9db3-737af51a6bc3 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:57.403506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d1b4b-cdfe-401f-b3a3-c1ff8817d9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:57.404113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d1b4b-cdfe-401f-b3a3-c1ff8817d9ea None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:57.404392 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d1b4b-cdfe-401f-b3a3-c1ff8817d9ea None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:57.404664 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d1b4b-cdfe-401f-b3a3-c1ff8817d9ea None admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:57.405331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d1b4b-cdfe-401f-b3a3-c1ff8817d9ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:57.405634 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d1b4b-cdfe-401f-b3a3-c1ff8817d9ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:57.407015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d1b4b-cdfe-401f-b3a3-c1ff8817d9ea None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:57.415939 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 631/1277] 10.222.0.62 () {56 vars in 1099 bytes} [Wed Sep 7 13:13:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:57.422707 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e88e4-25e1-4184-8e27-d68834778db8 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:57.525805 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e88e4-25e1-4184-8e27-d68834778db8 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:57.565589 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401f485-6b5a-49be-8b5e-109ffa621bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:57.566222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401f485-6b5a-49be-8b5e-109ffa621bf6 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:57.566499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401f485-6b5a-49be-8b5e-109ffa621bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:57.566771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401f485-6b5a-49be-8b5e-109ffa621bf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:57.578779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401f485-6b5a-49be-8b5e-109ffa621bf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:57.580904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401f485-6b5a-49be-8b5e-109ffa621bf6 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:57.597813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-a401f485-6b5a-49be-8b5e-109ffa621bf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:57.597813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:57.597813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:57.597813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:57.597813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:57.597813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:57.597813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:57.597813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:57.597813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:57.597813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:57.597813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:57.597813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:57.597813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:13:57.597813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:13:57.597813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:13:57.597813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:13:57.599789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:57.599789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:57.599789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:57.599789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:57.599789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:57.599789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:57.599789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:57.599789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:57.599789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:57.599789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:57.599789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:57.599789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:57.599789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:57.599789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:57.599789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:57.599789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:57.599789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:13:57.601568 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:13:57.601568 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:57.601568 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:13:57.601568 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 647/1278] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:57.605002 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d1b4b-cdfe-401f-b3a3-c1ff8817d9ea None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:57.709496 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d1b4b-cdfe-401f-b3a3-c1ff8817d9ea None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:57.745769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce60ac-c19a-4f8d-8eea-b57fa0aa90aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:57.746404 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce60ac-c19a-4f8d-8eea-b57fa0aa90aa None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:57.746684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce60ac-c19a-4f8d-8eea-b57fa0aa90aa None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:57.746954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce60ac-c19a-4f8d-8eea-b57fa0aa90aa None admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:57.747655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce60ac-c19a-4f8d-8eea-b57fa0aa90aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:57.747949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce60ac-c19a-4f8d-8eea-b57fa0aa90aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:57.749313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce60ac-c19a-4f8d-8eea-b57fa0aa90aa None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:57.760979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 632/1279] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:57.767087 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401f485-6b5a-49be-8b5e-109ffa621bf6 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:57.799096 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:57.872558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401f485-6b5a-49be-8b5e-109ffa621bf6 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:57.907792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c5e9d-b61e-447a-8df5-29b88d5637f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:57.908602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c5e9d-b61e-447a-8df5-29b88d5637f0 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:57.908878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c5e9d-b61e-447a-8df5-29b88d5637f0 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:57.909144 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c5e9d-b61e-447a-8df5-29b88d5637f0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:57.916394 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c5e9d-b61e-447a-8df5-29b88d5637f0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:57.917829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c5e9d-b61e-447a-8df5-29b88d5637f0 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:57.933201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-077c5e9d-b61e-447a-8df5-29b88d5637f0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:13:57.933201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:57.933201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:57.933201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:57.933201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:57.933201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:57.933201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:57.933201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:57.933201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:57.933201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:57.933201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:57.933201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:57.933201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 13:13:57.933201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 13:13:57.933201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 13:13:57.933201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:57.937811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:57.939003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:57.939003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:57.939003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 13:13:57.939003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 13:13:57.939003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 13:13:57.939003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 13:13:57.939003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 13:13:57.939003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 13:13:57.939003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 13:13:57.939003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:13:57.939003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 648/1280] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:57.941283 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce60ac-c19a-4f8d-8eea-b57fa0aa90aa None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:57.945321 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:58.043340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce60ac-c19a-4f8d-8eea-b57fa0aa90aa None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:58.077939 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcbcd1-6e5f-49b6-9447-64094a0b0f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:58.078639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcbcd1-6e5f-49b6-9447-64094a0b0f10 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:58.078919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcbcd1-6e5f-49b6-9447-64094a0b0f10 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:58.079225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcbcd1-6e5f-49b6-9447-64094a0b0f10 None admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:58.079960 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fcbcd1-6e5f-49b6-9447-64094a0b0f10 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:58.080292 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fcbcd1-6e5f-49b6-9447-64094a0b0f10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:58.081728 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fcbcd1-6e5f-49b6-9447-64094a0b0f10 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:58.099309 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 633/1281] 10.222.0.62 () {56 vars in 1143 bytes} [Wed Sep 7 13:13:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:58.105313 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c5e9d-b61e-447a-8df5-29b88d5637f0 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:58.202004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:58.212120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c5e9d-b61e-447a-8df5-29b88d5637f0 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:58.247812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ff3f4-f134-4f4c-bdbd-9e8e5fcdd031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:58.248411 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ff3f4-f134-4f4c-bdbd-9e8e5fcdd031 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:58.248689 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ff3f4-f134-4f4c-bdbd-9e8e5fcdd031 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:58.248958 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ff3f4-f134-4f4c-bdbd-9e8e5fcdd031 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:58.320425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ff3f4-f134-4f4c-bdbd-9e8e5fcdd031 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:58.321890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ff3f4-f134-4f4c-bdbd-9e8e5fcdd031 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:58.415980 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:58.426445 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 649/1282] 10.222.0.62 () {58 vars in 1282 bytes} [Wed Sep 7 13:13:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 13:13:58.432608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcbcd1-6e5f-49b6-9447-64094a0b0f10 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:58.495582 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:58.534848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcbcd1-6e5f-49b6-9447-64094a0b0f10 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:58.569920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f24ec-de40-4f52-a22e-b913671329ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:58.570563 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f24ec-de40-4f52-a22e-b913671329ec None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:58.570850 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f24ec-de40-4f52-a22e-b913671329ec None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:58.571159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f24ec-de40-4f52-a22e-b913671329ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:58.582111 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f24ec-de40-4f52-a22e-b913671329ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:58.583675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f24ec-de40-4f52-a22e-b913671329ec None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:58.601094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-3f9f24ec-de40-4f52-a22e-b913671329ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:58.601094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:58.601094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:58.601094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:58.601094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:58.601094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:58.601094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:58.601094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:58.601094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:58.601094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:58.601094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:58.601094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:58.601094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 13:13:58.601094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 13:13:58.601094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 13:13:58.601094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 13:13:58.602285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:58.602285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:58.602285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:58.602285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:58.602285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:58.602285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:58.602285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:58.602285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:58.602285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:58.602285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:58.602285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:58.602285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:58.602285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:58.602285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:58.602285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:58.602285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:58.602285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 13:13:58.603312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 13:13:58.603312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 13:13:58.603312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:13:58.603312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 634/1283] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:13:58.609097 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ff3f4-f134-4f4c-bdbd-9e8e5fcdd031 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:58.713846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ff3f4-f134-4f4c-bdbd-9e8e5fcdd031 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:58.751408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675504d-cfc8-419b-9fa1-f7b33a13ad73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:58.751938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675504d-cfc8-419b-9fa1-f7b33a13ad73 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:58.752233 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675504d-cfc8-419b-9fa1-f7b33a13ad73 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:58.752488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675504d-cfc8-419b-9fa1-f7b33a13ad73 None admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:58.753136 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675504d-cfc8-419b-9fa1-f7b33a13ad73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:58.753436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675504d-cfc8-419b-9fa1-f7b33a13ad73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:58.754780 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675504d-cfc8-419b-9fa1-f7b33a13ad73 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:58.768073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 650/1284] 10.222.0.62 () {56 vars in 1109 bytes} [Wed Sep 7 13:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:58.773929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f24ec-de40-4f52-a22e-b913671329ec None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:58.880242 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f24ec-de40-4f52-a22e-b913671329ec None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:58.899900 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:58.921681 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281880f-1822-41ec-8c82-57ffad0a1fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:58.922310 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281880f-1822-41ec-8c82-57ffad0a1fca None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:58.922592 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281880f-1822-41ec-8c82-57ffad0a1fca None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:58.922865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281880f-1822-41ec-8c82-57ffad0a1fca None admin] PATH_INFO: `/v3/projects/service` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:58.929583 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281880f-1822-41ec-8c82-57ffad0a1fca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:58.931335 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281880f-1822-41ec-8c82-57ffad0a1fca None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:58.943801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application [None req-1281880f-1822-41ec-8c82-57ffad0a1fca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:13:58.943801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:13:58.943801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:13:58.943801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:13:58.943801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:13:58.943801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:13:58.943801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:13:58.943801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:13:58.943801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:13:58.943801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:13:58.943801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:13:58.943801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:13:58.943801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 13:13:58.943801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 13:13:58.943801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 13:13:58.944840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 13:13:58.944840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:13:58.944840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:13:58.944840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:13:58.944840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:13:58.944840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:13:58.944840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:13:58.944840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:13:58.944840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application with Lock( Sep 07 13:13:58.944840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:13:58.944840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:13:58.944840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:13:58.944840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:13:58.944840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:13:58.944840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:13:58.944840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:13:58.944840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:13:58.945793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 13:13:58.945793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 13:13:58.945793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 13:13:58.945793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 13:13:58.945793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 13:13:58.945793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 13:13:58.945793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 13:13:58.945793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: ERROR keystone.server.flask.application Sep 07 13:13:58.945793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 635/1285] 10.222.0.62 () {56 vars in 1111 bytes} [Wed Sep 7 13:13:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 13:13:58.951989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675504d-cfc8-419b-9fa1-f7b33a13ad73 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:59.002700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:13:59.053396 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675504d-cfc8-419b-9fa1-f7b33a13ad73 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:59.087993 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec7812-0ac1-447e-a603-d249cc9f7195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:59.088686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec7812-0ac1-447e-a603-d249cc9f7195 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:59.088948 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec7812-0ac1-447e-a603-d249cc9f7195 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:59.089219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec7812-0ac1-447e-a603-d249cc9f7195 None admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:59.089895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec7812-0ac1-447e-a603-d249cc9f7195 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:59.090925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec7812-0ac1-447e-a603-d249cc9f7195 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:59.092356 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec7812-0ac1-447e-a603-d249cc9f7195 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:59.104179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 651/1286] 10.222.0.62 () {56 vars in 1148 bytes} [Wed Sep 7 13:13:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:13:59.111205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281880f-1822-41ec-8c82-57ffad0a1fca None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:13:59.213147 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281880f-1822-41ec-8c82-57ffad0a1fca None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:13:59.248736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a9776-52b5-48e1-b5fa-1d0a6a595bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:13:59.249338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a9776-52b5-48e1-b5fa-1d0a6a595bdd None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:13:59.249615 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a9776-52b5-48e1-b5fa-1d0a6a595bdd None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:13:59.249883 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a9776-52b5-48e1-b5fa-1d0a6a595bdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:13:59.250595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a9776-52b5-48e1-b5fa-1d0a6a595bdd None admin] RBAC: Adding query filter params (role.id=e151ac51ab124ad69365c08a0316c4ea, scope.project.id=4574cf3ff270485faf60594019f7e05c, user.id=bee90bf951f04c2e8e19ceeacb8cfcb2) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:13:59.250897 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a9776-52b5-48e1-b5fa-1d0a6a595bdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:13:59.252342 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a9776-52b5-48e1-b5fa-1d0a6a595bdd None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:13:59.259097 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 636/1287] 10.222.0.62 () {56 vars in 1374 bytes} [Wed Sep 7 13:13:59 2022] GET /identity/v3/role_assignments?user.id=bee90bf951f04c2e8e19ceeacb8cfcb2&scope.project.id=4574cf3ff270485faf60594019f7e05c&role.id=e151ac51ab124ad69365c08a0316c4ea => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:14:01.554768 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6102c-8639-4bf0-a15f-4b838ec98002 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:01.555408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6102c-8639-4bf0-a15f-4b838ec98002 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:01.555408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6102c-8639-4bf0-a15f-4b838ec98002 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:01.556680 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 652/1288] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:14:01.562780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8fc60-2ca7-447c-ad88-a01ca314d576 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:01.563316 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8fc60-2ca7-447c-ad88-a01ca314d576 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:01.566179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8fc60-2ca7-447c-ad88-a01ca314d576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:01.583829 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-a9a8fc60-2ca7-447c-ad88-a01ca314d576 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:01.585058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a9a8fc60-2ca7-447c-ad88-a01ca314d576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:01.620118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:01.670957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 637/1289] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:14:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:14:01.946807 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6102c-8639-4bf0-a15f-4b838ec98002 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:02.059843 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6102c-8639-4bf0-a15f-4b838ec98002 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:02.071450 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:02.096031 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c304f5-c317-4f5d-a6f5-8c3589b2bace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:02.096665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c304f5-c317-4f5d-a6f5-8c3589b2bace None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:02.096937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c304f5-c317-4f5d-a6f5-8c3589b2bace None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:02.097222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c304f5-c317-4f5d-a6f5-8c3589b2bace None admin] PATH_INFO: `/v3/services/placement` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:02.097795 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c304f5-c317-4f5d-a6f5-8c3589b2bace None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:02.099177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c304f5-c317-4f5d-a6f5-8c3589b2bace None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:02.108361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-00c304f5-c317-4f5d-a6f5-8c3589b2bace None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:14:02.108361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:14:02.108361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:14:02.108361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:14:02.108361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:14:02.108361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:14:02.108361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:14:02.108361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:14:02.108361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:14:02.108361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:14:02.108361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:14:02.108361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:14:02.108361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:14:02.108361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:14:02.108361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:14:02.108361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:14:02.108361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:14:02.109782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:14:02.109782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:14:02.109782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:14:02.109782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:14:02.109782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:14:02.109782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:14:02.109782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:14:02.109782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:14:02.109782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:14:02.109782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:14:02.109782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:14:02.109782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:14:02.109782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:14:02.109782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:14:02.109782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:14:02.109782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:14:02.110668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:14:02.110668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:14:02.110668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:14:02.110668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:14:02.110668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:14:02.110668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:14:02.110668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:14:02.110668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:14:02.110668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:14:02.110668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:14:02.110668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:14:02.110668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:14:02.110668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:14:02.110668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:14:02.110668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 653/1290] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 7 13:14:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:14:02.115905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8fc60-2ca7-447c-ad88-a01ca314d576 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:02.224335 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8fc60-2ca7-447c-ad88-a01ca314d576 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:02.265524 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadc834-fc76-41ec-8fb5-d980c3d5190a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:02.266136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadc834-fc76-41ec-8fb5-d980c3d5190a None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:02.266415 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadc834-fc76-41ec-8fb5-d980c3d5190a None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:02.266682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadc834-fc76-41ec-8fb5-d980c3d5190a None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:02.267378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadc834-fc76-41ec-8fb5-d980c3d5190a None admin] RBAC: Adding query filter params (name=placement) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:14:02.267674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadc834-fc76-41ec-8fb5-d980c3d5190a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:02.269031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadc834-fc76-41ec-8fb5-d980c3d5190a None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:02.274153 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 638/1291] 10.222.0.62 () {56 vars in 1116 bytes} [Wed Sep 7 13:14:02 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:14:02.283779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c304f5-c317-4f5d-a6f5-8c3589b2bace None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:02.396325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c304f5-c317-4f5d-a6f5-8c3589b2bace None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:02.431559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407b142-2033-4351-893d-77f2ac4cdf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:02.432163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407b142-2033-4351-893d-77f2ac4cdf54 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:02.432440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407b142-2033-4351-893d-77f2ac4cdf54 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:02.432768 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407b142-2033-4351-893d-77f2ac4cdf54 None admin] PATH_INFO: `/v3/services` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:02.433414 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407b142-2033-4351-893d-77f2ac4cdf54 None admin] RBAC: Adding query filter params (type=placement) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:14:02.433787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407b142-2033-4351-893d-77f2ac4cdf54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:02.436174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407b142-2033-4351-893d-77f2ac4cdf54 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:02.441076 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 654/1292] 10.222.0.62 () {56 vars in 1116 bytes} [Wed Sep 7 13:14:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:14:04.631619 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12c8cb-acd5-44cb-acfb-c5c21f1fff71 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:04.632190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12c8cb-acd5-44cb-acfb-c5c21f1fff71 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:04.632190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12c8cb-acd5-44cb-acfb-c5c21f1fff71 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:04.633608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 639/1293] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:14:04.639232 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a47f5c-1985-465d-a9e3-2e282634c03e None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:04.639698 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a47f5c-1985-465d-a9e3-2e282634c03e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:04.639822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a47f5c-1985-465d-a9e3-2e282634c03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:04.659219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-d7a47f5c-1985-465d-a9e3-2e282634c03e None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:04.660542 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d7a47f5c-1985-465d-a9e3-2e282634c03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:04.691551 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:04.731817 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 655/1294] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:14:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:14:05.019789 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12c8cb-acd5-44cb-acfb-c5c21f1fff71 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:05.055939 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:05.139873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12c8cb-acd5-44cb-acfb-c5c21f1fff71 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:05.175329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006af0bd-d793-421f-90fd-605634700403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:05.175959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006af0bd-d793-421f-90fd-605634700403 None admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:05.176241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006af0bd-d793-421f-90fd-605634700403 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:05.176517 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006af0bd-d793-421f-90fd-605634700403 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:05.177171 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006af0bd-d793-421f-90fd-605634700403 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:05.178544 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006af0bd-d793-421f-90fd-605634700403 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:05.311513 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 640/1295] 10.222.0.62 () {60 vars in 1141 bytes} [Wed Sep 7 13:14:05 2022] POST /identity/v3/services => generated 246 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:14:07.544161 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f4a62-88e7-4c09-b373-63a91dee226a None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:07.544688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f4a62-88e7-4c09-b373-63a91dee226a None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:07.544688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f4a62-88e7-4c09-b373-63a91dee226a None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:07.546080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 656/1296] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:14:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:14:07.551652 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61d189-bd49-43dd-a946-4c787bc46448 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:07.552109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61d189-bd49-43dd-a946-4c787bc46448 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:07.552243 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61d189-bd49-43dd-a946-4c787bc46448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:07.570887 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-2e61d189-bd49-43dd-a946-4c787bc46448 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:07.572127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2e61d189-bd49-43dd-a946-4c787bc46448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:07.656323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 641/1297] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:14:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:14:07.931437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f4a62-88e7-4c09-b373-63a91dee226a None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:08.041345 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f4a62-88e7-4c09-b373-63a91dee226a None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:08.077513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4e8eb-cf41-4df1-9f09-d1c750c69bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:08.078148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4e8eb-cf41-4df1-9f09-d1c750c69bbc None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:08.078425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4e8eb-cf41-4df1-9f09-d1c750c69bbc None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:08.078717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4e8eb-cf41-4df1-9f09-d1c750c69bbc None admin] PATH_INFO: `/v3/services/placement` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:08.079313 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4e8eb-cf41-4df1-9f09-d1c750c69bbc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:08.080676 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4e8eb-cf41-4df1-9f09-d1c750c69bbc None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:08.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-6dc4e8eb-cf41-4df1-9f09-d1c750c69bbc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:14:08.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:14:08.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:14:08.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:14:08.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:14:08.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:14:08.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:14:08.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:14:08.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:14:08.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:14:08.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:14:08.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:14:08.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:14:08.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:14:08.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:14:08.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:14:08.090544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:14:08.092779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:14:08.092779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:14:08.092779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:14:08.092779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:14:08.092779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:14:08.092779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:14:08.092779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:14:08.092779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:14:08.092779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:14:08.092779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:14:08.092779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:14:08.092779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:14:08.092779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:14:08.092779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:14:08.092779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:14:08.092779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:14:08.094933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:14:08.094933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:14:08.094933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:14:08.094933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:14:08.094933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:14:08.094933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:14:08.094933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:14:08.094933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:14:08.094933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:14:08.094933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:14:08.094933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:14:08.094933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:14:08.094933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:14:08.094933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:14:08.094933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 657/1298] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 7 13:14:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:14:08.098270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61d189-bd49-43dd-a946-4c787bc46448 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:08.118020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:08.200125 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61d189-bd49-43dd-a946-4c787bc46448 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:08.237969 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e6095-91e5-4200-845f-c8c676e34206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:08.238576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e6095-91e5-4200-845f-c8c676e34206 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:08.238853 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e6095-91e5-4200-845f-c8c676e34206 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:08.239181 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e6095-91e5-4200-845f-c8c676e34206 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:08.239892 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e6095-91e5-4200-845f-c8c676e34206 None admin] RBAC: Adding query filter params (name=placement) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:14:08.240093 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e6095-91e5-4200-845f-c8c676e34206 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:08.241449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e6095-91e5-4200-845f-c8c676e34206 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:08.246689 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 642/1299] 10.222.0.62 () {56 vars in 1116 bytes} [Wed Sep 7 13:14:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:14:08.253163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4e8eb-cf41-4df1-9f09-d1c750c69bbc None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:08.296876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:08.359630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4e8eb-cf41-4df1-9f09-d1c750c69bbc None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:08.394088 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab463a65-81b1-46a6-8b56-6b35d9e7e056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:08.394693 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab463a65-81b1-46a6-8b56-6b35d9e7e056 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:08.394966 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab463a65-81b1-46a6-8b56-6b35d9e7e056 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:08.395301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab463a65-81b1-46a6-8b56-6b35d9e7e056 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:08.395994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab463a65-81b1-46a6-8b56-6b35d9e7e056 None admin] RBAC: Adding query filter params (interface=public, service_id=e09896f8e4bb40aa82f12ccec1637470, region_id=RegionOne) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:14:08.396296 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab463a65-81b1-46a6-8b56-6b35d9e7e056 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:08.397640 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab463a65-81b1-46a6-8b56-6b35d9e7e056 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:08.403524 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 658/1300] 10.222.0.62 () {56 vars in 1251 bytes} [Wed Sep 7 13:14:08 2022] GET /identity/v3/endpoints?service_id=e09896f8e4bb40aa82f12ccec1637470&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:14:08.412166 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e6095-91e5-4200-845f-c8c676e34206 None admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:08.522573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:08.541293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e6095-91e5-4200-845f-c8c676e34206 None admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:08.580607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a12f5-76e3-48c2-b990-e7b2d610aa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:08.581325 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a12f5-76e3-48c2-b990-e7b2d610aa9c None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:08.581510 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a12f5-76e3-48c2-b990-e7b2d610aa9c None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:08.581774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a12f5-76e3-48c2-b990-e7b2d610aa9c None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:08.582386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a12f5-76e3-48c2-b990-e7b2d610aa9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:08.583785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a12f5-76e3-48c2-b990-e7b2d610aa9c None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:08.591696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 643/1301] 10.222.0.62 () {56 vars in 1087 bytes} [Wed Sep 7 13:14:08 2022] GET /identity/v3/services => generated 2191 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:14:10.812484 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cf038-2b35-4e31-b4e4-7bf72e1099ef None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:10.813158 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cf038-2b35-4e31-b4e4-7bf72e1099ef None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:10.813404 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cf038-2b35-4e31-b4e4-7bf72e1099ef None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:10.816073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 659/1302] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:14:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:14:10.821877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7db019-dece-4d3c-816e-0c14e13babd0 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:10.822418 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7db019-dece-4d3c-816e-0c14e13babd0 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:10.822418 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7db019-dece-4d3c-816e-0c14e13babd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:10.842851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-ef7db019-dece-4d3c-816e-0c14e13babd0 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:14:10.844090 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ef7db019-dece-4d3c-816e-0c14e13babd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:14:10.917482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 644/1303] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:14:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 13:14:11.191064 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cf038-2b35-4e31-b4e4-7bf72e1099ef None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:11.294536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:11.301405 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cf038-2b35-4e31-b4e4-7bf72e1099ef None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:11.339106 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abff551-2ab7-45f5-b347-4d29bdcd57b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:11.339741 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abff551-2ab7-45f5-b347-4d29bdcd57b3 None admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:11.340027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abff551-2ab7-45f5-b347-4d29bdcd57b3 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:11.340318 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abff551-2ab7-45f5-b347-4d29bdcd57b3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:11.340902 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abff551-2ab7-45f5-b347-4d29bdcd57b3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:11.342305 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abff551-2ab7-45f5-b347-4d29bdcd57b3 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:11.351267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application [None req-8abff551-2ab7-45f5-b347-4d29bdcd57b3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:14:11.351267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:14:11.351267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 13:14:11.351267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 13:14:11.351267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 13:14:11.351267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 13:14:11.351267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 13:14:11.351267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:14:11.351267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:14:11.351267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:14:11.351267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 13:14:11.351267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:14:11.351267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 13:14:11.351267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 13:14:11.351267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 13:14:11.351267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 13:14:11.351267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 13:14:11.355182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 13:14:11.355182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 13:14:11.355182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 13:14:11.355182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 13:14:11.355182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 13:14:11.355182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 13:14:11.355182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 13:14:11.355182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 13:14:11.355182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 13:14:11.355182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 13:14:11.355182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 13:14:11.355182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 13:14:11.355182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 13:14:11.355182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 13:14:11.355182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 13:14:11.355182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 13:14:11.356124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 13:14:11.356124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application with Lock( Sep 07 13:14:11.356124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 13:14:11.356124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self._enter() Sep 07 13:14:11.356124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 13:14:11.356124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 13:14:11.356124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 13:14:11.356124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application return self.creator() Sep 07 13:14:11.356124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 13:14:11.356124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application created_value = creator( Sep 07 13:14:11.356124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 13:14:11.356124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 13:14:11.356124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 13:14:11.356124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: ERROR keystone.server.flask.application Sep 07 13:14:11.356124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 660/1304] 10.222.0.62 () {56 vars in 1117 bytes} [Wed Sep 7 13:14:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 13:14:11.358736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7db019-dece-4d3c-816e-0c14e13babd0 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:11.437701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:11.462070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7db019-dece-4d3c-816e-0c14e13babd0 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:11.497943 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4154e95-5143-49fa-bad5-4ee457b2c501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:11.498551 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4154e95-5143-49fa-bad5-4ee457b2c501 None admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:11.498831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4154e95-5143-49fa-bad5-4ee457b2c501 None admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:11.499146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4154e95-5143-49fa-bad5-4ee457b2c501 None admin] PATH_INFO: `/v3/services` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:11.499781 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4154e95-5143-49fa-bad5-4ee457b2c501 None admin] RBAC: Adding query filter params (name=placement) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:14:11.500083 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4154e95-5143-49fa-bad5-4ee457b2c501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:11.501443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4154e95-5143-49fa-bad5-4ee457b2c501 None admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:11.506615 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 645/1305] 10.222.0.62 () {56 vars in 1116 bytes} [Wed Sep 7 13:14:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:14:11.513432 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abff551-2ab7-45f5-b347-4d29bdcd57b3 None admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:14:11.625826 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abff551-2ab7-45f5-b347-4d29bdcd57b3 None admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:14:11.661677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b1e3e-a745-4c8a-8464-b1c5342eab93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:14:11.662283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b1e3e-a745-4c8a-8464-b1c5342eab93 None admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:14:11.662564 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b1e3e-a745-4c8a-8464-b1c5342eab93 None admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:14:11.662984 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b1e3e-a745-4c8a-8464-b1c5342eab93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:14:11.663553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6b1e3e-a745-4c8a-8464-b1c5342eab93 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:14:11.664912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6b1e3e-a745-4c8a-8464-b1c5342eab93 None admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:14:11.818861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 661/1306] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:14:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:14:21.554390 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=30 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:14:21.814365 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:27.931509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc140c-78cb-4bda-8aff-ed7c1abd4f79 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:27.932117 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc140c-78cb-4bda-8aff-ed7c1abd4f79 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:27.932117 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc140c-78cb-4bda-8aff-ed7c1abd4f79 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:27.941470 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0c821-6f0b-41ce-a037-a7a81b8e41ba None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:27.942032 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0c821-6f0b-41ce-a037-a7a81b8e41ba None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:27.942032 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0c821-6f0b-41ce-a037-a7a81b8e41ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:27.947179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 646/1307] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:23:27 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:23:28.065106 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-98e0c821-6f0b-41ce-a037-a7a81b8e41ba None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:23:28.066746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-98e0c821-6f0b-41ce-a037-a7a81b8e41ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:28.226736 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 662/1308] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:23:28.538509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2d7e6-ffef-4279-bdb7-0f77a0d88c9f None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:28.539008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2d7e6-ffef-4279-bdb7-0f77a0d88c9f None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:28.539514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2d7e6-ffef-4279-bdb7-0f77a0d88c9f None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:28.540959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 647/1309] 10.222.0.62 () {54 vars in 965 bytes} [Wed Sep 7 13:23:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:23:28.548193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790ed9d-f50c-4aad-a8bb-694ccb254d8c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:28.548492 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790ed9d-f50c-4aad-a8bb-694ccb254d8c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:28.548649 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790ed9d-f50c-4aad-a8bb-694ccb254d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:28.653237 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-8790ed9d-f50c-4aad-a8bb-694ccb254d8c None None] MFA Rules not processed for user `5acdbdb6f23b4f10bcbab0387ea67ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:23:28.654451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8790ed9d-f50c-4aad-a8bb-694ccb254d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:28.684222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:28.775509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 663/1310] 10.222.0.62 () {58 vars in 1062 bytes} [Wed Sep 7 13:23:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:23:28.785950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2d7e6-ffef-4279-bdb7-0f77a0d88c9f None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:23:28.788161 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d1f2d7e6-ffef-4279-bdb7-0f77a0d88c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:28.933926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2d7e6-ffef-4279-bdb7-0f77a0d88c9f None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:23:29.034746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5cfaf-7d7d-4440-b7d1-b9c388b9e673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:23:29.035631 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5cfaf-7d7d-4440-b7d1-b9c388b9e673 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:29.035967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5cfaf-7d7d-4440-b7d1-b9c388b9e673 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:29.036306 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5cfaf-7d7d-4440-b7d1-b9c388b9e673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:29.038207 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-68e5cfaf-7d7d-4440-b7d1-b9c388b9e673 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:29.105729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:29.177789 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5cfaf-7d7d-4440-b7d1-b9c388b9e673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:23:29.180599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5cfaf-7d7d-4440-b7d1-b9c388b9e673 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:23:29.215314 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 648/1311] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:23:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:23:31.945745 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466a039-a6d0-4494-b9f0-2249452b9e21 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:31.946272 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466a039-a6d0-4494-b9f0-2249452b9e21 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:31.946272 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466a039-a6d0-4494-b9f0-2249452b9e21 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:31.947664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 664/1312] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:23:31.954223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884473ac-93ce-4952-b8d4-0fb05435aa79 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:31.954737 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884473ac-93ce-4952-b8d4-0fb05435aa79 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:31.954844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884473ac-93ce-4952-b8d4-0fb05435aa79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:32.031013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-884473ac-93ce-4952-b8d4-0fb05435aa79 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:23:32.032299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-884473ac-93ce-4952-b8d4-0fb05435aa79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:32.134312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 649/1313] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:23:32.443746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466a039-a6d0-4494-b9f0-2249452b9e21 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:23:32.445454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0466a039-a6d0-4494-b9f0-2249452b9e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:32.563962 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466a039-a6d0-4494-b9f0-2249452b9e21 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:23:32.644724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:32.647585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbba39-021a-4408-bbe5-711d46f287cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:23:32.648208 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbba39-021a-4408-bbe5-711d46f287cb service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:32.648576 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbba39-021a-4408-bbe5-711d46f287cb service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:32.648848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbba39-021a-4408-bbe5-711d46f287cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:32.650504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d4cbba39-021a-4408-bbe5-711d46f287cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:32.773488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbba39-021a-4408-bbe5-711d46f287cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:23:32.775036 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbba39-021a-4408-bbe5-711d46f287cb service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:23:32.799633 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 665/1314] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:23:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:23:35.612753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129c1db-45ee-4436-9e1d-beb9f31fac96 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:35.613406 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129c1db-45ee-4436-9e1d-beb9f31fac96 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:35.613406 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129c1db-45ee-4436-9e1d-beb9f31fac96 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:35.614654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 650/1315] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:23:35.620495 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acdca7-15ea-40a2-8782-6169a188b4f3 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:35.621001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acdca7-15ea-40a2-8782-6169a188b4f3 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:35.621276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acdca7-15ea-40a2-8782-6169a188b4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:35.695491 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-76acdca7-15ea-40a2-8782-6169a188b4f3 None None] MFA Rules not processed for user `58c09285a32c4e3886cdd7b86f345a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:23:35.696740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-76acdca7-15ea-40a2-8782-6169a188b4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:35.757958 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:35.803207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 666/1316] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:23:35 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:23:36.114957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7499d0d-8d80-4c49-a624-0760e9affea0 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:36.115268 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7499d0d-8d80-4c49-a624-0760e9affea0 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:36.115547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7499d0d-8d80-4c49-a624-0760e9affea0 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:36.116974 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 651/1317] 10.222.0.62 () {54 vars in 965 bytes} [Wed Sep 7 13:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:23:36.123776 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfdfe5-6913-43cb-b4d0-70b24ba40656 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:36.124122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfdfe5-6913-43cb-b4d0-70b24ba40656 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:36.124533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfdfe5-6913-43cb-b4d0-70b24ba40656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:36.204458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-77cfdfe5-6913-43cb-b4d0-70b24ba40656 None None] MFA Rules not processed for user `5acdbdb6f23b4f10bcbab0387ea67ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:23:36.205681 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-77cfdfe5-6913-43cb-b4d0-70b24ba40656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:36.290996 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 667/1318] 10.222.0.62 () {58 vars in 1062 bytes} [Wed Sep 7 13:23:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:23:36.300877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7499d0d-8d80-4c49-a624-0760e9affea0 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:23:36.302556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c7499d0d-8d80-4c49-a624-0760e9affea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:36.313934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:36.413519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7499d0d-8d80-4c49-a624-0760e9affea0 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:23:36.491576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d91adb-f8f8-4216-bd5a-a319790209de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:23:36.492187 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91adb-f8f8-4216-bd5a-a319790209de service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:36.492476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91adb-f8f8-4216-bd5a-a319790209de service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:36.492779 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91adb-f8f8-4216-bd5a-a319790209de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:36.494426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-16d91adb-f8f8-4216-bd5a-a319790209de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:36.560214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:36.615896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d91adb-f8f8-4216-bd5a-a319790209de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:23:36.617355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d91adb-f8f8-4216-bd5a-a319790209de service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:23:36.637537 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 652/1319] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:23:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:23:40.162334 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffecb4-96e8-46c1-9885-ca77df206b23 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:40.163062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffecb4-96e8-46c1-9885-ca77df206b23 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:40.163062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffecb4-96e8-46c1-9885-ca77df206b23 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:40.164279 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 668/1320] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:23:40.170268 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304e80c-0b8f-4973-ac76-538817ff31a3 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:40.170954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304e80c-0b8f-4973-ac76-538817ff31a3 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:40.171081 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304e80c-0b8f-4973-ac76-538817ff31a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:40.242523 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-5304e80c-0b8f-4973-ac76-538817ff31a3 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:23:40.243787 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5304e80c-0b8f-4973-ac76-538817ff31a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:40.349873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 653/1321] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:23:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:23:40.643422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffecb4-96e8-46c1-9885-ca77df206b23 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:23:40.645099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-03ffecb4-96e8-46c1-9885-ca77df206b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:40.731173 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:40.832366 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffecb4-96e8-46c1-9885-ca77df206b23 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:23:40.931075 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a87d0a-6c3f-4c6a-879b-158dd667b1f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:23:40.931692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a87d0a-6c3f-4c6a-879b-158dd667b1f7 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:40.931970 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a87d0a-6c3f-4c6a-879b-158dd667b1f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:40.932242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a87d0a-6c3f-4c6a-879b-158dd667b1f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:40.933984 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f1a87d0a-6c3f-4c6a-879b-158dd667b1f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:41.042456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:41.057741 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a87d0a-6c3f-4c6a-879b-158dd667b1f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:23:41.059315 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a87d0a-6c3f-4c6a-879b-158dd667b1f7 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:23:41.082856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 669/1322] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:23:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:23:44.655059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bb316-43c7-48da-937c-54bf0be5ddb2 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:44.655830 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bb316-43c7-48da-937c-54bf0be5ddb2 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:44.655971 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62bb316-43c7-48da-937c-54bf0be5ddb2 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:44.658222 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 654/1323] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:23:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:23:44.667190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ae7d1-98fb-4cf4-94df-d760231dc631 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:44.667748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ae7d1-98fb-4cf4-94df-d760231dc631 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:44.668317 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ae7d1-98fb-4cf4-94df-d760231dc631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:44.769877 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-e61ae7d1-98fb-4cf4-94df-d760231dc631 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:23:44.770994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e61ae7d1-98fb-4cf4-94df-d760231dc631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:44.890349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 670/1324] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:23:45.212611 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bb316-43c7-48da-937c-54bf0be5ddb2 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:23:45.214209 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d62bb316-43c7-48da-937c-54bf0be5ddb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:45.237370 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:45.355120 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62bb316-43c7-48da-937c-54bf0be5ddb2 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:23:45.491637 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987787a9-b990-4fd4-8751-e8d63087b655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:23:45.492257 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987787a9-b990-4fd4-8751-e8d63087b655 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:45.501849 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987787a9-b990-4fd4-8751-e8d63087b655 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:45.501849 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987787a9-b990-4fd4-8751-e8d63087b655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:45.501849 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-987787a9-b990-4fd4-8751-e8d63087b655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:45.630167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:45.688943 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987787a9-b990-4fd4-8751-e8d63087b655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:23:45.690823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987787a9-b990-4fd4-8751-e8d63087b655 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:23:45.712510 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 655/1325] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:23:48.323892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a629e5-d944-4beb-8f88-0f19f5422381 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:48.324432 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a629e5-d944-4beb-8f88-0f19f5422381 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:48.324432 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a629e5-d944-4beb-8f88-0f19f5422381 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:48.325854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 671/1326] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:23:48.332637 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4da5b-bcd5-40ff-842e-b5dd54afe632 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:48.333202 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4da5b-bcd5-40ff-842e-b5dd54afe632 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:48.333202 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4da5b-bcd5-40ff-842e-b5dd54afe632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:48.427128 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-b8a4da5b-bcd5-40ff-842e-b5dd54afe632 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:23:48.428395 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b8a4da5b-bcd5-40ff-842e-b5dd54afe632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:48.574148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 656/1327] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:23:48.863423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a629e5-d944-4beb-8f88-0f19f5422381 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:23:48.865149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c0a629e5-d944-4beb-8f88-0f19f5422381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:48.952290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:49.012760 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a629e5-d944-4beb-8f88-0f19f5422381 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:23:49.111642 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5a7c7-b52e-4ece-b78e-a057ba4e0c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:23:49.112211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5a7c7-b52e-4ece-b78e-a057ba4e0c3a service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:49.112490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5a7c7-b52e-4ece-b78e-a057ba4e0c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:49.112785 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5a7c7-b52e-4ece-b78e-a057ba4e0c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:49.114514 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a9b5a7c7-b52e-4ece-b78e-a057ba4e0c3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:49.278261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:49.282836 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5a7c7-b52e-4ece-b78e-a057ba4e0c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:23:49.285696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5a7c7-b52e-4ece-b78e-a057ba4e0c3a service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:23:49.313200 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 672/1328] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:23:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:23:51.769924 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f0c24-37d3-41fd-910b-b714f44b3473 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:51.770440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f0c24-37d3-41fd-910b-b714f44b3473 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:51.770716 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f0c24-37d3-41fd-910b-b714f44b3473 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:51.773138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 657/1329] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:23:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:23:51.779964 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92040c75-78ae-4d1f-811c-d6c5dea9c653 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:51.780507 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92040c75-78ae-4d1f-811c-d6c5dea9c653 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:51.780507 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92040c75-78ae-4d1f-811c-d6c5dea9c653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:51.858785 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-92040c75-78ae-4d1f-811c-d6c5dea9c653 None None] MFA Rules not processed for user `58c09285a32c4e3886cdd7b86f345a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:23:51.860070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-92040c75-78ae-4d1f-811c-d6c5dea9c653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:51.976466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 673/1330] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:23:51 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:23:52.306673 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f0c24-37d3-41fd-910b-b714f44b3473 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:23:52.308322 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-372f0c24-37d3-41fd-910b-b714f44b3473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:52.359631 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:52.516246 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f0c24-37d3-41fd-910b-b714f44b3473 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:23:52.674031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8cfc83-91d6-4732-af54-f6a8cb78e301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:23:52.674960 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cfc83-91d6-4732-af54-f6a8cb78e301 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:52.675484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cfc83-91d6-4732-af54-f6a8cb78e301 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:52.675991 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cfc83-91d6-4732-af54-f6a8cb78e301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:52.678494 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2c8cfc83-91d6-4732-af54-f6a8cb78e301 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:52.812587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:52.865683 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8cfc83-91d6-4732-af54-f6a8cb78e301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:23:52.867926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8cfc83-91d6-4732-af54-f6a8cb78e301 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:23:52.916282 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 658/1331] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:23:52 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:23:56.075187 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d1943-ba55-4b7d-aae0-32f31fe024bf None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:56.076560 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d1943-ba55-4b7d-aae0-32f31fe024bf None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:56.076732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d1943-ba55-4b7d-aae0-32f31fe024bf None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:56.078645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 674/1332] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:23:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:23:56.086911 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf411855-d809-4085-aebe-0d4345b2da23 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:56.087554 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf411855-d809-4085-aebe-0d4345b2da23 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:56.087898 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf411855-d809-4085-aebe-0d4345b2da23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:56.186174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-cf411855-d809-4085-aebe-0d4345b2da23 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:23:56.188129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-cf411855-d809-4085-aebe-0d4345b2da23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:56.356060 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 659/1333] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:23:56.674626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d1943-ba55-4b7d-aae0-32f31fe024bf None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:23:56.676549 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-016d1943-ba55-4b7d-aae0-32f31fe024bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:56.791798 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:56.835984 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d1943-ba55-4b7d-aae0-32f31fe024bf None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:23:56.974534 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c349f-33a9-4232-89e5-be0510b7fabc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:23:56.975439 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c349f-33a9-4232-89e5-be0510b7fabc service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:23:56.975876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c349f-33a9-4232-89e5-be0510b7fabc service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:23:56.976454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c349f-33a9-4232-89e5-be0510b7fabc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:23:56.979922 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7a1c349f-33a9-4232-89e5-be0510b7fabc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:23:57.170599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c349f-33a9-4232-89e5-be0510b7fabc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:23:57.172371 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c349f-33a9-4232-89e5-be0510b7fabc service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:23:57.190267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:23:57.200886 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 675/1334] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:23:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:24:00.129150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff7f28-5b08-450b-89ec-5ff5f4d4df50 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:00.130032 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff7f28-5b08-450b-89ec-5ff5f4d4df50 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:00.130032 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff7f28-5b08-450b-89ec-5ff5f4d4df50 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:00.131525 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 660/1335] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:24:00.138220 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd209ef-d9d8-4a32-887a-31572a1e23ca None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:00.139120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd209ef-d9d8-4a32-887a-31572a1e23ca None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:00.139325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd209ef-d9d8-4a32-887a-31572a1e23ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:00.252117 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-3bd209ef-d9d8-4a32-887a-31572a1e23ca None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:00.254186 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-3bd209ef-d9d8-4a32-887a-31572a1e23ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:00.430666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 676/1336] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:00.877000 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff7f28-5b08-450b-89ec-5ff5f4d4df50 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:00.880100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-45ff7f28-5b08-450b-89ec-5ff5f4d4df50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:00.966913 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:01.105903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff7f28-5b08-450b-89ec-5ff5f4d4df50 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:01.223741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb10749-4083-4d15-a181-554732c11935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:01.224581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb10749-4083-4d15-a181-554732c11935 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:01.224744 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb10749-4083-4d15-a181-554732c11935 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:01.224998 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb10749-4083-4d15-a181-554732c11935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:01.227008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-8bb10749-4083-4d15-a181-554732c11935 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:01.355703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:01.386408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb10749-4083-4d15-a181-554732c11935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:01.388824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb10749-4083-4d15-a181-554732c11935 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:01.417312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 661/1337] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:04.629093 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d958e9d-fab2-490c-a1c3-12cc7ae2eb19 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:04.629720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d958e9d-fab2-490c-a1c3-12cc7ae2eb19 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:04.629720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d958e9d-fab2-490c-a1c3-12cc7ae2eb19 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:04.631202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 677/1338] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:24:04.637243 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb6ac2-e52a-4226-bfe7-a6797e7aace0 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:04.638569 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb6ac2-e52a-4226-bfe7-a6797e7aace0 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:04.638846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb6ac2-e52a-4226-bfe7-a6797e7aace0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:04.711845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-6afb6ac2-e52a-4226-bfe7-a6797e7aace0 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:04.713097 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6afb6ac2-e52a-4226-bfe7-a6797e7aace0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:04.819510 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 662/1339] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:24:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:05.103930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d958e9d-fab2-490c-a1c3-12cc7ae2eb19 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:05.105479 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7d958e9d-fab2-490c-a1c3-12cc7ae2eb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:05.183770 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:05.212643 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d958e9d-fab2-490c-a1c3-12cc7ae2eb19 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:05.291650 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39048e-d5ce-4281-a25d-4fd0e004b033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:05.292260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39048e-d5ce-4281-a25d-4fd0e004b033 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:05.292535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39048e-d5ce-4281-a25d-4fd0e004b033 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:05.292801 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39048e-d5ce-4281-a25d-4fd0e004b033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:05.294447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-5a39048e-d5ce-4281-a25d-4fd0e004b033 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:05.406448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39048e-d5ce-4281-a25d-4fd0e004b033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:05.407797 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39048e-d5ce-4281-a25d-4fd0e004b033 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:05.425920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 678/1340] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:10.081963 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d4f35-a13b-46dc-ba18-b0ebfa4c67a8 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:10.082513 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d4f35-a13b-46dc-ba18-b0ebfa4c67a8 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:10.082513 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d4f35-a13b-46dc-ba18-b0ebfa4c67a8 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:10.083898 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 663/1341] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:24:10.089739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67792229-6734-4b8b-afc9-8ba3f422d2a8 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:10.090778 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67792229-6734-4b8b-afc9-8ba3f422d2a8 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:10.091209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67792229-6734-4b8b-afc9-8ba3f422d2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:10.097308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:10.173364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-67792229-6734-4b8b-afc9-8ba3f422d2a8 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:10.174647 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-67792229-6734-4b8b-afc9-8ba3f422d2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:10.306938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 679/1342] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:24:10.601369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d4f35-a13b-46dc-ba18-b0ebfa4c67a8 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:10.604530 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f04d4f35-a13b-46dc-ba18-b0ebfa4c67a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:10.666638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:10.732721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d4f35-a13b-46dc-ba18-b0ebfa4c67a8 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:10.815741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43946c7f-0451-474c-8829-85745022df6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:10.816353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43946c7f-0451-474c-8829-85745022df6f service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:10.816628 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43946c7f-0451-474c-8829-85745022df6f service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:10.816894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43946c7f-0451-474c-8829-85745022df6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:10.818662 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-43946c7f-0451-474c-8829-85745022df6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:10.922997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:10.941182 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43946c7f-0451-474c-8829-85745022df6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:10.942636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43946c7f-0451-474c-8829-85745022df6f service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:10.962648 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 664/1343] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:14.632360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f09ce-5db9-4fdd-ba22-d6d929f4cb72 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:14.632934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f09ce-5db9-4fdd-ba22-d6d929f4cb72 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:14.633044 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f09ce-5db9-4fdd-ba22-d6d929f4cb72 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:14.634469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 680/1344] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:24:14.640596 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed2b67-4d73-4fdf-9f48-7c0e0754c132 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:14.641090 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed2b67-4d73-4fdf-9f48-7c0e0754c132 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:14.641193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed2b67-4d73-4fdf-9f48-7c0e0754c132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:14.730136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-82ed2b67-4d73-4fdf-9f48-7c0e0754c132 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:14.731998 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-82ed2b67-4d73-4fdf-9f48-7c0e0754c132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:14.853770 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 665/1345] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:24:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:15.147119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f09ce-5db9-4fdd-ba22-d6d929f4cb72 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:15.148661 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-be8f09ce-5db9-4fdd-ba22-d6d929f4cb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:15.252340 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:15.276402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f09ce-5db9-4fdd-ba22-d6d929f4cb72 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:15.361290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f691c5-7256-4b88-858c-35227290c787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:15.364714 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f691c5-7256-4b88-858c-35227290c787 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:15.364714 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f691c5-7256-4b88-858c-35227290c787 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:15.364714 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f691c5-7256-4b88-858c-35227290c787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:15.364714 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e9f691c5-7256-4b88-858c-35227290c787 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:15.518026 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f691c5-7256-4b88-858c-35227290c787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:15.519577 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f691c5-7256-4b88-858c-35227290c787 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:15.542109 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 681/1346] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:20.305796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda29e49-fedb-447e-9c13-5310acb11fe0 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:20.308268 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda29e49-fedb-447e-9c13-5310acb11fe0 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:20.308905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda29e49-fedb-447e-9c13-5310acb11fe0 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:20.310997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 666/1347] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:24:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:24:20.317300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286468fd-ca19-4a02-9180-edb1379e3eed None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:20.317825 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286468fd-ca19-4a02-9180-edb1379e3eed None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:20.317911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286468fd-ca19-4a02-9180-edb1379e3eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:20.343529 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:20.405336 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-286468fd-ca19-4a02-9180-edb1379e3eed None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:20.406586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-286468fd-ca19-4a02-9180-edb1379e3eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:20.527863 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 682/1348] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:20.816867 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda29e49-fedb-447e-9c13-5310acb11fe0 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:20.818639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-bda29e49-fedb-447e-9c13-5310acb11fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:20.896588 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:20.948900 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda29e49-fedb-447e-9c13-5310acb11fe0 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:21.038413 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ea6c3-cd0e-4867-a4a8-16a07da411bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:21.039069 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ea6c3-cd0e-4867-a4a8-16a07da411bb service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:21.039364 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ea6c3-cd0e-4867-a4a8-16a07da411bb service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:21.039632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ea6c3-cd0e-4867-a4a8-16a07da411bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:21.042319 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-eb9ea6c3-cd0e-4867-a4a8-16a07da411bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:21.181923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:21.187212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ea6c3-cd0e-4867-a4a8-16a07da411bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:21.188702 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ea6c3-cd0e-4867-a4a8-16a07da411bb service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:21.209565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 667/1349] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:26.782573 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6366dfd-f885-4fbd-9b02-e25d21daa077 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:26.783631 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6366dfd-f885-4fbd-9b02-e25d21daa077 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:26.783631 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6366dfd-f885-4fbd-9b02-e25d21daa077 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:26.786177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 683/1350] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 7 13:24:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:24:26.793544 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425ff23-1b01-4fe1-bf74-54a76a5984f6 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:26.794178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425ff23-1b01-4fe1-bf74-54a76a5984f6 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:26.795101 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425ff23-1b01-4fe1-bf74-54a76a5984f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:26.931310 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-3425ff23-1b01-4fe1-bf74-54a76a5984f6 None None] MFA Rules not processed for user `bee90bf951f04c2e8e19ceeacb8cfcb2`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:26.932801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-3425ff23-1b01-4fe1-bf74-54a76a5984f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:27.100162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 668/1351] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 7 13:24:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:30.525281 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=59 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:31.866510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2477e26-2857-4294-b671-f291b327b144 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:31.867250 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2477e26-2857-4294-b671-f291b327b144 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:31.867250 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2477e26-2857-4294-b671-f291b327b144 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:31.868625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 684/1352] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 7 13:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:24:31.874553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf23d74-f425-4c29-980d-26a02a77df9f None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:31.875076 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf23d74-f425-4c29-980d-26a02a77df9f None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:31.875076 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf23d74-f425-4c29-980d-26a02a77df9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:31.938068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:31.963019 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-fdf23d74-f425-4c29-980d-26a02a77df9f None None] MFA Rules not processed for user `bee90bf951f04c2e8e19ceeacb8cfcb2`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:31.964331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-fdf23d74-f425-4c29-980d-26a02a77df9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:32.094592 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 669/1353] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 7 13:24:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:33.429089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19cc07-807e-4dc7-b4b1-f4b98af7bc20 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:33.429660 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19cc07-807e-4dc7-b4b1-f4b98af7bc20 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:33.429759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19cc07-807e-4dc7-b4b1-f4b98af7bc20 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:33.431364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 685/1354] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 7 13:24:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:24:33.437556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caba53-f0e6-42b0-8a0d-148dd007f11f None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:33.438082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caba53-f0e6-42b0-8a0d-148dd007f11f None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:33.438184 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caba53-f0e6-42b0-8a0d-148dd007f11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:33.525076 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-98caba53-f0e6-42b0-8a0d-148dd007f11f None None] MFA Rules not processed for user `bee90bf951f04c2e8e19ceeacb8cfcb2`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:33.526374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-98caba53-f0e6-42b0-8a0d-148dd007f11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:33.586817 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:33.667864 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 670/1355] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 7 13:24:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:37.704360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e71e17-9bed-4f82-9f5e-1fdb3b75336f None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:37.705116 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e71e17-9bed-4f82-9f5e-1fdb3b75336f None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:37.705116 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e71e17-9bed-4f82-9f5e-1fdb3b75336f None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:37.706318 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 686/1356] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:24:37.712061 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483d18f-6e28-4ca9-8dc8-a618ea1f3105 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:37.720109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483d18f-6e28-4ca9-8dc8-a618ea1f3105 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:37.720109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483d18f-6e28-4ca9-8dc8-a618ea1f3105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:37.796507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-8483d18f-6e28-4ca9-8dc8-a618ea1f3105 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:37.797708 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-8483d18f-6e28-4ca9-8dc8-a618ea1f3105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:37.923516 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 671/1357] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:38.232122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d1ad7-fac8-4f36-8f98-fa5b067c6f80 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:38.233083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d1ad7-fac8-4f36-8f98-fa5b067c6f80 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:38.233869 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d1ad7-fac8-4f36-8f98-fa5b067c6f80 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:38.236260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 687/1358] 10.222.0.62 () {54 vars in 962 bytes} [Wed Sep 7 13:24:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:24:38.248193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2b258-a489-4dee-8f0c-18b9fd41b1cb None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:38.248929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2b258-a489-4dee-8f0c-18b9fd41b1cb None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:38.249321 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2b258-a489-4dee-8f0c-18b9fd41b1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:38.268655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:38.353287 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-bcc2b258-a489-4dee-8f0c-18b9fd41b1cb None None] MFA Rules not processed for user `97c363046db444cbb3d4346066f9ad42`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:38.355842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-bcc2b258-a489-4dee-8f0c-18b9fd41b1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:38.493820 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 672/1359] 10.222.0.62 () {58 vars in 1059 bytes} [Wed Sep 7 13:24:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:38.503446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d1ad7-fac8-4f36-8f98-fa5b067c6f80 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:38.506216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d97d1ad7-fac8-4f36-8f98-fa5b067c6f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:38.705564 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d1ad7-fac8-4f36-8f98-fa5b067c6f80 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:38.795298 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:38.851820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bd29d-922e-4fcb-87fe-1c92c585cdfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:38.852449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bd29d-922e-4fcb-87fe-1c92c585cdfa service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:38.852839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bd29d-922e-4fcb-87fe-1c92c585cdfa service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:38.853021 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bd29d-922e-4fcb-87fe-1c92c585cdfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:38.854686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b60bd29d-922e-4fcb-87fe-1c92c585cdfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:38.971664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bd29d-922e-4fcb-87fe-1c92c585cdfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:38.973591 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bd29d-922e-4fcb-87fe-1c92c585cdfa service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:38.997056 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 688/1360] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:41.560494 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8860b6e-e58f-4aba-8816-2d5ffcd7488b None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:41.561106 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8860b6e-e58f-4aba-8816-2d5ffcd7488b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:41.561106 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8860b6e-e58f-4aba-8816-2d5ffcd7488b None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:41.562510 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 673/1361] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:24:41.569130 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98df8c-9289-4d8c-9ccc-43c4f892d14f None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:41.569801 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98df8c-9289-4d8c-9ccc-43c4f892d14f None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:41.569801 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98df8c-9289-4d8c-9ccc-43c4f892d14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:41.649500 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-3a98df8c-9289-4d8c-9ccc-43c4f892d14f None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:41.650776 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-3a98df8c-9289-4d8c-9ccc-43c4f892d14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:41.700086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:41.771738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 689/1362] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:24:42.199484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8860b6e-e58f-4aba-8816-2d5ffcd7488b None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:42.200788 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e8860b6e-e58f-4aba-8816-2d5ffcd7488b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:42.312458 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:42.410363 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8860b6e-e58f-4aba-8816-2d5ffcd7488b None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:42.567874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2276be3-fc33-44e5-8d47-1e990e0770d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:42.568639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2276be3-fc33-44e5-8d47-1e990e0770d1 service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:42.568743 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2276be3-fc33-44e5-8d47-1e990e0770d1 service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:42.569088 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2276be3-fc33-44e5-8d47-1e990e0770d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:42.570949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f2276be3-fc33-44e5-8d47-1e990e0770d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:42.601476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:42.682149 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2276be3-fc33-44e5-8d47-1e990e0770d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:42.684879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2276be3-fc33-44e5-8d47-1e990e0770d1 service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:42.705059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 674/1363] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:45.619352 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9755a-149e-43fc-8232-90aeaf8027cb None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:45.620099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9755a-149e-43fc-8232-90aeaf8027cb None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:45.621308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9755a-149e-43fc-8232-90aeaf8027cb None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:45.626592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 690/1364] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:24:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:24:45.633717 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea52f3-f5a9-47bb-aef6-94058bb18fb9 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:45.634568 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea52f3-f5a9-47bb-aef6-94058bb18fb9 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:45.634786 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea52f3-f5a9-47bb-aef6-94058bb18fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:45.727221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c5ea52f3-f5a9-47bb-aef6-94058bb18fb9 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:45.728806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c5ea52f3-f5a9-47bb-aef6-94058bb18fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:45.883563 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:45.891242 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 675/1365] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:46.220801 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ef6f0-01d0-4cad-95b4-e221dfcebe6a None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:46.221441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ef6f0-01d0-4cad-95b4-e221dfcebe6a None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:46.221927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ef6f0-01d0-4cad-95b4-e221dfcebe6a None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:46.223702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 691/1366] 10.222.0.62 () {54 vars in 962 bytes} [Wed Sep 7 13:24:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:24:46.230814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37331ad-a8f1-41db-8447-39181ed58d7f None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:46.231738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37331ad-a8f1-41db-8447-39181ed58d7f None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:46.232019 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37331ad-a8f1-41db-8447-39181ed58d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:46.359082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-d37331ad-a8f1-41db-8447-39181ed58d7f None None] MFA Rules not processed for user `97c363046db444cbb3d4346066f9ad42`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:46.360375 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d37331ad-a8f1-41db-8447-39181ed58d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:46.520456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 676/1367] 10.222.0.62 () {58 vars in 1059 bytes} [Wed Sep 7 13:24:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:46.530499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ef6f0-01d0-4cad-95b4-e221dfcebe6a None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:46.534695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-870ef6f0-01d0-4cad-95b4-e221dfcebe6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:46.757148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ef6f0-01d0-4cad-95b4-e221dfcebe6a None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:46.782131 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:46.830714 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61ba5a-54da-4720-b476-7d8098191191 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:46.832075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61ba5a-54da-4720-b476-7d8098191191 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:46.832377 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61ba5a-54da-4720-b476-7d8098191191 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:46.834103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 677/1368] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:24:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:24:46.840925 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902aaa47-ff7c-4975-91b2-8cc7bcccc1ca None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:46.841233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902aaa47-ff7c-4975-91b2-8cc7bcccc1ca None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:46.841747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902aaa47-ff7c-4975-91b2-8cc7bcccc1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:46.933646 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:46.942533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-902aaa47-ff7c-4975-91b2-8cc7bcccc1ca None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:46.943843 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-902aaa47-ff7c-4975-91b2-8cc7bcccc1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:46.959660 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711031e-823e-41e5-ad14-58df2fbd9522 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:46.960306 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711031e-823e-41e5-ad14-58df2fbd9522 service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:46.960581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711031e-823e-41e5-ad14-58df2fbd9522 service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:46.960878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711031e-823e-41e5-ad14-58df2fbd9522 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:46.962689 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b711031e-823e-41e5-ad14-58df2fbd9522 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:47.091777 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711031e-823e-41e5-ad14-58df2fbd9522 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:47.093560 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711031e-823e-41e5-ad14-58df2fbd9522 service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:47.095176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 678/1369] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:24:47.117380 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 692/1370] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:47.431088 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41255edc-20b5-4182-9d1b-004c5be33957 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:47.431593 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41255edc-20b5-4182-9d1b-004c5be33957 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:47.431946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41255edc-20b5-4182-9d1b-004c5be33957 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:47.434544 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 679/1371] 10.222.0.62 () {54 vars in 964 bytes} [Wed Sep 7 13:24:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:24:47.440934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edecf4e-0a12-434d-9c40-8411952326e9 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:47.441306 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edecf4e-0a12-434d-9c40-8411952326e9 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:47.441893 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edecf4e-0a12-434d-9c40-8411952326e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:47.485171 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:47.581383 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-3edecf4e-0a12-434d-9c40-8411952326e9 None None] MFA Rules not processed for user `fc2fa0fa46b842ceb1cac437166b8aea`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:47.582798 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-3edecf4e-0a12-434d-9c40-8411952326e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:47.695444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 693/1372] 10.222.0.62 () {58 vars in 1061 bytes} [Wed Sep 7 13:24:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:47.706632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41255edc-20b5-4182-9d1b-004c5be33957 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:47.711324 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-41255edc-20b5-4182-9d1b-004c5be33957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:47.888570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41255edc-20b5-4182-9d1b-004c5be33957 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:47.931991 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:47.991424 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aee726-5057-40de-82fb-daa2d4c84a4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:47.992276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aee726-5057-40de-82fb-daa2d4c84a4a service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:47.992276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aee726-5057-40de-82fb-daa2d4c84a4a service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:47.992864 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aee726-5057-40de-82fb-daa2d4c84a4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:47.994353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b5aee726-5057-40de-82fb-daa2d4c84a4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:48.150148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aee726-5057-40de-82fb-daa2d4c84a4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:48.152591 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aee726-5057-40de-82fb-daa2d4c84a4a service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:48.176058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 680/1373] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:50.098184 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e571-b165-4140-8ba0-8e7dd9642aa7 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:50.098736 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e571-b165-4140-8ba0-8e7dd9642aa7 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:50.098912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e571-b165-4140-8ba0-8e7dd9642aa7 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:50.100709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 694/1374] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:24:50.106814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe3099-d861-4d09-a91e-129679f6d83e None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:50.107338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe3099-d861-4d09-a91e-129679f6d83e None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:50.107447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe3099-d861-4d09-a91e-129679f6d83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:50.197649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-bdfe3099-d861-4d09-a91e-129679f6d83e None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:50.199643 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-bdfe3099-d861-4d09-a91e-129679f6d83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:50.207738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:50.317950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 681/1375] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:50.617447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881e571-b165-4140-8ba0-8e7dd9642aa7 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:50.619147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8881e571-b165-4140-8ba0-8e7dd9642aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:50.764994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:50.822189 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881e571-b165-4140-8ba0-8e7dd9642aa7 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:51.039056 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfbda9-a027-4903-a0e5-94a5becd1b9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:51.039738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfbda9-a027-4903-a0e5-94a5becd1b9f service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:51.040106 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfbda9-a027-4903-a0e5-94a5becd1b9f service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:51.040380 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfbda9-a027-4903-a0e5-94a5becd1b9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:51.043011 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-54bfbda9-a027-4903-a0e5-94a5becd1b9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:51.147830 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:51.165881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bfbda9-a027-4903-a0e5-94a5becd1b9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:51.168417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bfbda9-a027-4903-a0e5-94a5becd1b9f service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:51.190704 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 695/1376] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:53.990460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332d633-613d-4309-a497-10198c2b1510 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:53.990460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332d633-613d-4309-a497-10198c2b1510 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:53.990460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332d633-613d-4309-a497-10198c2b1510 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:53.990460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 682/1377] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:24:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:24:53.991938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3159c-471c-41d2-849b-c9869924ea54 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:53.991938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3159c-471c-41d2-849b-c9869924ea54 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:53.991938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3159c-471c-41d2-849b-c9869924ea54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:53.991938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-fbd3159c-471c-41d2-849b-c9869924ea54 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:53.991938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-fbd3159c-471c-41d2-849b-c9869924ea54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:54.009479 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 696/1378] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:24:54.345689 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332d633-613d-4309-a497-10198c2b1510 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:54.347419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-4332d633-613d-4309-a497-10198c2b1510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:54.555692 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:54.567815 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332d633-613d-4309-a497-10198c2b1510 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:54.707480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfe444-d779-4641-8729-700c24d199c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:54.708209 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfe444-d779-4641-8729-700c24d199c0 service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:54.708577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfe444-d779-4641-8729-700c24d199c0 service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:54.708915 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfe444-d779-4641-8729-700c24d199c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:54.710743 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-8fbfe444-d779-4641-8729-700c24d199c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:54.828238 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfe444-d779-4641-8729-700c24d199c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:54.831850 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfe444-d779-4641-8729-700c24d199c0 service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:54.853495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:54.871689 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 683/1379] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:24:57.569206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ee0fc-e0b6-4f0f-bb55-2a8f8532dbd7 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:57.569850 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ee0fc-e0b6-4f0f-bb55-2a8f8532dbd7 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:57.570008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ee0fc-e0b6-4f0f-bb55-2a8f8532dbd7 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:57.571819 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 697/1380] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:24:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:24:57.581796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5ba89-fd70-418c-b6e9-5424509ef7a2 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:57.582729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5ba89-fd70-418c-b6e9-5424509ef7a2 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:57.582729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5ba89-fd70-418c-b6e9-5424509ef7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:57.668982 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-3ae5ba89-fd70-418c-b6e9-5424509ef7a2 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:24:57.670837 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-3ae5ba89-fd70-418c-b6e9-5424509ef7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:57.822100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 684/1381] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:24:58.166942 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ee0fc-e0b6-4f0f-bb55-2a8f8532dbd7 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:24:58.168645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e21ee0fc-e0b6-4f0f-bb55-2a8f8532dbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:58.269274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:58.408506 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ee0fc-e0b6-4f0f-bb55-2a8f8532dbd7 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:24:58.578982 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15c8d4-8866-4ed8-83e3-c35f414aa246 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:24:58.579830 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15c8d4-8866-4ed8-83e3-c35f414aa246 service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:24:58.580004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15c8d4-8866-4ed8-83e3-c35f414aa246 service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:24:58.580147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15c8d4-8866-4ed8-83e3-c35f414aa246 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:24:58.581850 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-fe15c8d4-8866-4ed8-83e3-c35f414aa246 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:24:58.606634 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:24:58.699011 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15c8d4-8866-4ed8-83e3-c35f414aa246 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:24:58.700502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15c8d4-8866-4ed8-83e3-c35f414aa246 service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:24:58.722121 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 698/1382] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:24:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:01.783793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd760c4-1c9d-4f98-81b7-dfed71f86309 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:01.784613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd760c4-1c9d-4f98-81b7-dfed71f86309 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:01.784613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd760c4-1c9d-4f98-81b7-dfed71f86309 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:01.786618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 685/1383] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:25:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:25:01.795260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb6878-68e9-4d1d-9529-e97f2761cd2a None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:01.796411 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb6878-68e9-4d1d-9529-e97f2761cd2a None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:01.796411 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb6878-68e9-4d1d-9529-e97f2761cd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:01.915697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b1bb6878-68e9-4d1d-9529-e97f2761cd2a None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:01.917293 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b1bb6878-68e9-4d1d-9529-e97f2761cd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:02.140529 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:02.153932 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 699/1384] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:02.577124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd760c4-1c9d-4f98-81b7-dfed71f86309 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:02.579613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ccd760c4-1c9d-4f98-81b7-dfed71f86309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:02.689231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:02.810216 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd760c4-1c9d-4f98-81b7-dfed71f86309 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:02.993603 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ab93a-0a38-4354-b309-3f8c4917fe40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:02.995112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ab93a-0a38-4354-b309-3f8c4917fe40 service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:02.995582 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ab93a-0a38-4354-b309-3f8c4917fe40 service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:02.996247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ab93a-0a38-4354-b309-3f8c4917fe40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:02.999316 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-289ab93a-0a38-4354-b309-3f8c4917fe40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:03.053629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:03.139503 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ab93a-0a38-4354-b309-3f8c4917fe40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:03.141582 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ab93a-0a38-4354-b309-3f8c4917fe40 service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:03.163003 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 686/1385] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:04.746153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5e875-eadd-46cb-a881-4fcb9e34c595 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:04.747020 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5e875-eadd-46cb-a881-4fcb9e34c595 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:04.747020 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5e875-eadd-46cb-a881-4fcb9e34c595 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:04.748103 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 700/1386] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:25:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:25:04.756567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fa924-f1b5-4216-9d3d-c2a80c0d3083 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:04.757401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fa924-f1b5-4216-9d3d-c2a80c0d3083 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:04.757401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fa924-f1b5-4216-9d3d-c2a80c0d3083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:04.843087 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-348fa924-f1b5-4216-9d3d-c2a80c0d3083 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:04.844326 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-348fa924-f1b5-4216-9d3d-c2a80c0d3083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:04.973696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 687/1387] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:05.314663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38706264-ed7a-4ede-8b1a-0fa7716c4100 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.315012 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38706264-ed7a-4ede-8b1a-0fa7716c4100 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.315197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38706264-ed7a-4ede-8b1a-0fa7716c4100 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.316515 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 701/1388] 10.222.0.62 () {54 vars in 961 bytes} [Wed Sep 7 13:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:25:05.324340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9efde-ad83-4048-ab64-2501e96b6783 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.324842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9efde-ad83-4048-ab64-2501e96b6783 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.325327 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9efde-ad83-4048-ab64-2501e96b6783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.356783 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:05.461841 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-acf9efde-ad83-4048-ab64-2501e96b6783 None None] MFA Rules not processed for user `d1a48805d95640489d7ce3333dd687b9`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:05.463328 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-acf9efde-ad83-4048-ab64-2501e96b6783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:05.555998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11667b59-5c29-4879-b3cd-480314f1fa9b None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.557133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11667b59-5c29-4879-b3cd-480314f1fa9b None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.557595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11667b59-5c29-4879-b3cd-480314f1fa9b None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.559970 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 702/1389] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:25:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:25:05.566848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97784c9b-a343-447b-a124-6d3a524eefb1 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.567161 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97784c9b-a343-447b-a124-6d3a524eefb1 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.567537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97784c9b-a343-447b-a124-6d3a524eefb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.592027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 688/1390] 10.222.0.62 () {58 vars in 1058 bytes} [Wed Sep 7 13:25:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:05.603827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9efde-ad83-4048-ab64-2501e96b6783 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:05.613178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9efde-ad83-4048-ab64-2501e96b6783 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:05.658910 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-97784c9b-a343-447b-a124-6d3a524eefb1 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:05.661236 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-97784c9b-a343-447b-a124-6d3a524eefb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:05.706161 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0fa00-9f0d-441a-818c-26b8f72105d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1a48805d95640489d7ce3333dd687b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:05.706780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0fa00-9f0d-441a-818c-26b8f72105d7 service glance] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:05.707090 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0fa00-9f0d-441a-818c-26b8f72105d7 service glance] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:05.707365 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0fa00-9f0d-441a-818c-26b8f72105d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:05.709056 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ebf0fa00-9f0d-441a-818c-26b8f72105d7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:05.747279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:05.837595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 703/1391] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:25:05.891374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0fa00-9f0d-441a-818c-26b8f72105d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:05.894020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0fa00-9f0d-441a-818c-26b8f72105d7 service glance] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:05.927764 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 689/1392] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:25:06.148433 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97784c9b-a343-447b-a124-6d3a524eefb1 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:06.149722 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-97784c9b-a343-447b-a124-6d3a524eefb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:06.248966 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:06.315951 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97784c9b-a343-447b-a124-6d3a524eefb1 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:06.501699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a615cfa-5db9-440f-8693-02c60bb55331 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:06.502309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a615cfa-5db9-440f-8693-02c60bb55331 service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:06.502599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a615cfa-5db9-440f-8693-02c60bb55331 service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:06.502955 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a615cfa-5db9-440f-8693-02c60bb55331 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:06.505289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-6a615cfa-5db9-440f-8693-02c60bb55331 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:06.622848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:06.655521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a615cfa-5db9-440f-8693-02c60bb55331 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:06.659086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a615cfa-5db9-440f-8693-02c60bb55331 service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:06.693340 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 704/1393] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:25:06.945143 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b266be5-976b-4cbe-8ae4-c6e531232b42 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:06.945919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b266be5-976b-4cbe-8ae4-c6e531232b42 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:06.945919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b266be5-976b-4cbe-8ae4-c6e531232b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:07.056682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-5b266be5-976b-4cbe-8ae4-c6e531232b42 None None] MFA Rules not processed for user `8121ba4cf2cf44d9a5a55975ea85928c`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:07.058807 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5b266be5-976b-4cbe-8ae4-c6e531232b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:07.179753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:07.187078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 690/1394] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 7 13:25:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:25:07.241419 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bef6e3-860f-4b54-a801-f95a3c12053c None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:07.242309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bef6e3-860f-4b54-a801-f95a3c12053c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:07.242309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bef6e3-860f-4b54-a801-f95a3c12053c None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:07.245655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 705/1395] 10.222.0.62 () {54 vars in 943 bytes} [Wed Sep 7 13:25:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:25:07.255499 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa095d9-15bc-4346-a98f-be568accdbcd None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:07.255984 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa095d9-15bc-4346-a98f-be568accdbcd None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:07.256497 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa095d9-15bc-4346-a98f-be568accdbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:07.397400 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-caa095d9-15bc-4346-a98f-be568accdbcd None None] MFA Rules not processed for user `5cedc7091d2f46b5be68818d26399e9b`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:07.398549 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-caa095d9-15bc-4346-a98f-be568accdbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:07.536868 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 691/1396] 10.222.0.62 () {58 vars in 1040 bytes} [Wed Sep 7 13:25:07 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:07.548979 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bef6e3-860f-4b54-a801-f95a3c12053c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:07.549690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-45bef6e3-860f-4b54-a801-f95a3c12053c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:07.752459 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bef6e3-860f-4b54-a801-f95a3c12053c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:07.823939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:07.881505 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75067db-e3f8-4cbb-9e5d-64fcf07252e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cedc7091d2f46b5be68818d26399e9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:07.882241 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75067db-e3f8-4cbb-9e5d-64fcf07252e4 service swift] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:07.882737 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75067db-e3f8-4cbb-9e5d-64fcf07252e4 service swift] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:07.883152 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75067db-e3f8-4cbb-9e5d-64fcf07252e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:07.885571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b75067db-e3f8-4cbb-9e5d-64fcf07252e4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:07.987479 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75067db-e3f8-4cbb-9e5d-64fcf07252e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:07.989032 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75067db-e3f8-4cbb-9e5d-64fcf07252e4 service swift] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:07.998821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 706/1397] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 7 13:25:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:25:09.479405 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa0742-ce43-4006-a634-ab520e8089bc None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:09.480461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa0742-ce43-4006-a634-ab520e8089bc None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:09.480461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa0742-ce43-4006-a634-ab520e8089bc None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:09.484400 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 692/1398] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:25:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:25:09.491381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2424eb-8aa9-4e98-8e1d-086fdc33470d None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:09.492212 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2424eb-8aa9-4e98-8e1d-086fdc33470d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:09.492396 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2424eb-8aa9-4e98-8e1d-086fdc33470d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:09.636126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-1e2424eb-8aa9-4e98-8e1d-086fdc33470d None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:09.638218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1e2424eb-8aa9-4e98-8e1d-086fdc33470d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:09.717084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:09.783711 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 707/1399] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:10.085382 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa0742-ce43-4006-a634-ab520e8089bc None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:10.086919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-afaa0742-ce43-4006-a634-ab520e8089bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:10.170669 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:10.298636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa0742-ce43-4006-a634-ab520e8089bc None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:10.508668 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30473a6-6f9d-46a2-8861-859c61827816 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:10.509464 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30473a6-6f9d-46a2-8861-859c61827816 service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:10.509627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30473a6-6f9d-46a2-8861-859c61827816 service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:10.509896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30473a6-6f9d-46a2-8861-859c61827816 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:10.512004 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d30473a6-6f9d-46a2-8861-859c61827816 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:10.533626 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:10.644450 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30473a6-6f9d-46a2-8861-859c61827816 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:10.646195 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30473a6-6f9d-46a2-8861-859c61827816 service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:10.671365 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 693/1400] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:13.197574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a471a67-7ecc-4446-9be0-06ad5f57bd7c None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:13.197574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a471a67-7ecc-4446-9be0-06ad5f57bd7c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:13.197574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a471a67-7ecc-4446-9be0-06ad5f57bd7c None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:13.203026 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 708/1401] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:25:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:25:13.204305 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f4f80-7331-4594-b3b0-4d092c60b62e None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:13.204927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f4f80-7331-4594-b3b0-4d092c60b62e None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:13.205531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f4f80-7331-4594-b3b0-4d092c60b62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:13.300921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-178f4f80-7331-4594-b3b0-4d092c60b62e None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:13.302664 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-178f4f80-7331-4594-b3b0-4d092c60b62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:13.456041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:13.471275 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 694/1402] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:13.841357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a471a67-7ecc-4446-9be0-06ad5f57bd7c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:13.844299 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9a471a67-7ecc-4446-9be0-06ad5f57bd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:14.083526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a471a67-7ecc-4446-9be0-06ad5f57bd7c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:14.133040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:14.290232 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df7c46-e48d-4dc7-8bb3-1a1aae9799a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:14.291311 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df7c46-e48d-4dc7-8bb3-1a1aae9799a2 service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:14.291741 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df7c46-e48d-4dc7-8bb3-1a1aae9799a2 service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:14.292244 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df7c46-e48d-4dc7-8bb3-1a1aae9799a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:14.295498 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d9df7c46-e48d-4dc7-8bb3-1a1aae9799a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:14.450510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df7c46-e48d-4dc7-8bb3-1a1aae9799a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:14.454966 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df7c46-e48d-4dc7-8bb3-1a1aae9799a2 service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:14.485544 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 709/1403] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:17.086605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d347ce-2175-4ff7-a473-9798885af20c None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:17.087897 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d347ce-2175-4ff7-a473-9798885af20c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:17.087897 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d347ce-2175-4ff7-a473-9798885af20c None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:17.089361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 695/1404] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:25:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:25:17.095167 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a894ce-78ca-4acd-91eb-8d07a8068445 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:17.095791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a894ce-78ca-4acd-91eb-8d07a8068445 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:17.095791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a894ce-78ca-4acd-91eb-8d07a8068445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:17.169587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:17.217423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-13a894ce-78ca-4acd-91eb-8d07a8068445 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:17.219577 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-13a894ce-78ca-4acd-91eb-8d07a8068445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:17.382351 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 710/1405] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:17.783268 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d347ce-2175-4ff7-a473-9798885af20c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:17.785590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-65d347ce-2175-4ff7-a473-9798885af20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:18.065880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d347ce-2175-4ff7-a473-9798885af20c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:18.151711 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:18.248453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89204c00-97a4-4367-8446-95bdd2a85c7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:18.249577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89204c00-97a4-4367-8446-95bdd2a85c7d service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:18.250079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89204c00-97a4-4367-8446-95bdd2a85c7d service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:18.250560 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89204c00-97a4-4367-8446-95bdd2a85c7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:18.253796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-89204c00-97a4-4367-8446-95bdd2a85c7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:18.438120 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89204c00-97a4-4367-8446-95bdd2a85c7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:18.441035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89204c00-97a4-4367-8446-95bdd2a85c7d service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:18.468480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 696/1406] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 686 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:21.113292 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fb3b2-23dc-4ba1-a976-c4a882a59f87 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:21.113938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fb3b2-23dc-4ba1-a976-c4a882a59f87 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:21.113938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fb3b2-23dc-4ba1-a976-c4a882a59f87 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:21.119549 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 711/1407] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:25:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:25:21.121642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6a47d-407c-4290-8b8b-1998674d81d0 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:21.122129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6a47d-407c-4290-8b8b-1998674d81d0 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:21.122466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6a47d-407c-4290-8b8b-1998674d81d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:21.233701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-70c6a47d-407c-4290-8b8b-1998674d81d0 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:21.235072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-70c6a47d-407c-4290-8b8b-1998674d81d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:21.282756 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:21.419404 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 697/1408] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:21.713521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fb3b2-23dc-4ba1-a976-c4a882a59f87 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:21.715795 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d22fb3b2-23dc-4ba1-a976-c4a882a59f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:21.883579 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:21.958521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fb3b2-23dc-4ba1-a976-c4a882a59f87 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:22.137883 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dcc5e5-cad1-46ef-a6f3-1cdf8483830b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:22.139435 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dcc5e5-cad1-46ef-a6f3-1cdf8483830b service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:22.139775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dcc5e5-cad1-46ef-a6f3-1cdf8483830b service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:22.140119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dcc5e5-cad1-46ef-a6f3-1cdf8483830b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:22.141917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-71dcc5e5-cad1-46ef-a6f3-1cdf8483830b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:22.276837 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dcc5e5-cad1-46ef-a6f3-1cdf8483830b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:22.279061 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dcc5e5-cad1-46ef-a6f3-1cdf8483830b service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:22.301866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:22.319897 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 712/1409] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:25:31.406691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=30 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:32.339094 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=13 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:35.391249 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f58ed7-0740-4878-8be4-4a460526d67e None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:35.391814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f58ed7-0740-4878-8be4-4a460526d67e None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:35.391814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f58ed7-0740-4878-8be4-4a460526d67e None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:35.392956 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 698/1410] 10.222.0.62 () {54 vars in 918 bytes} [Wed Sep 7 13:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:25:35.399896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b55464-5900-45f8-907b-b209b2a4aa32 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:35.400570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b55464-5900-45f8-907b-b209b2a4aa32 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:35.400781 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b55464-5900-45f8-907b-b209b2a4aa32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:35.535832 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b7b55464-5900-45f8-907b-b209b2a4aa32 None None] MFA Rules not processed for user `bee90bf951f04c2e8e19ceeacb8cfcb2`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:35.537244 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b7b55464-5900-45f8-907b-b209b2a4aa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:35.704396 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 713/1411] 10.222.0.62 () {58 vars in 1015 bytes} [Wed Sep 7 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:35.723206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c2b75-f2d6-4d83-9e7a-4149b0edd972 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:35.723621 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c2b75-f2d6-4d83-9e7a-4149b0edd972 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:35.723945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c2b75-f2d6-4d83-9e7a-4149b0edd972 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:35.725426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 699/1412] 10.222.0.62 () {54 vars in 918 bytes} [Wed Sep 7 13:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:25:35.732245 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a3b8b-a83f-4816-9ecd-7027971cdf1a None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:35.732526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a3b8b-a83f-4816-9ecd-7027971cdf1a None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:35.732790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a3b8b-a83f-4816-9ecd-7027971cdf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:35.823862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:35.827801 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-fb4a3b8b-a83f-4816-9ecd-7027971cdf1a None None] MFA Rules not processed for user `97c363046db444cbb3d4346066f9ad42`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:35.829108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-fb4a3b8b-a83f-4816-9ecd-7027971cdf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:36.061012 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 714/1413] 10.222.0.62 () {58 vars in 1015 bytes} [Wed Sep 7 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:25:38.861540 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9579ac3-db33-429b-a414-f1fd967fb91b None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:38.862167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9579ac3-db33-429b-a414-f1fd967fb91b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:38.862167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9579ac3-db33-429b-a414-f1fd967fb91b None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:38.863532 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:25:38.869277 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d930d4-e9c4-434d-b64c-824ce5a7e52a None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:38.869828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d930d4-e9c4-434d-b64c-824ce5a7e52a None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:38.869828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d930d4-e9c4-434d-b64c-824ce5a7e52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:38.947333 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-91d930d4-e9c4-434d-b64c-824ce5a7e52a None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:38.948585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-91d930d4-e9c4-434d-b64c-824ce5a7e52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:39.013634 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:39.059336 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:39.369686 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9579ac3-db33-429b-a414-f1fd967fb91b None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:39.371636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e9579ac3-db33-429b-a414-f1fd967fb91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:39.533239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9579ac3-db33-429b-a414-f1fd967fb91b None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:39.620302 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:39.670207 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa793f-2939-4e2e-9d73-6b36c30039bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:39.670827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa793f-2939-4e2e-9d73-6b36c30039bd service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:39.671173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa793f-2939-4e2e-9d73-6b36c30039bd service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:39.671443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa793f-2939-4e2e-9d73-6b36c30039bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:39.673121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-69fa793f-2939-4e2e-9d73-6b36c30039bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:39.776462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa793f-2939-4e2e-9d73-6b36c30039bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:39.778071 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa793f-2939-4e2e-9d73-6b36c30039bd service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:39.798103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 701/1416] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:41.390756 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31149a6-02fa-4123-b3dd-7d549d8a6cad None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:41.391338 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31149a6-02fa-4123-b3dd-7d549d8a6cad None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:41.391338 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31149a6-02fa-4123-b3dd-7d549d8a6cad None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:41.392642 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:25:41.400653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb57047-09c3-4d10-ba2b-cb0629707af4 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:41.401227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb57047-09c3-4d10-ba2b-cb0629707af4 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:41.401227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb57047-09c3-4d10-ba2b-cb0629707af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:41.482774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-5bb57047-09c3-4d10-ba2b-cb0629707af4 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:41.484061 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5bb57047-09c3-4d10-ba2b-cb0629707af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:41.514385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:41.617072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:41.792774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31149a6-02fa-4123-b3dd-7d549d8a6cad None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:41.794336 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f31149a6-02fa-4123-b3dd-7d549d8a6cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:41.962740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31149a6-02fa-4123-b3dd-7d549d8a6cad None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:42.006522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:42.111041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8403b3-4a87-4b35-9ad3-4aa43b0f96fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:42.111924 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8403b3-4a87-4b35-9ad3-4aa43b0f96fd service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:42.112385 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8403b3-4a87-4b35-9ad3-4aa43b0f96fd service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:42.112740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8403b3-4a87-4b35-9ad3-4aa43b0f96fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:42.115786 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8e8403b3-4a87-4b35-9ad3-4aa43b0f96fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:42.229517 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8403b3-4a87-4b35-9ad3-4aa43b0f96fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:42.231463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8403b3-4a87-4b35-9ad3-4aa43b0f96fd service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:42.252044 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 717/1419] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:43.494379 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c2ce4-2cc4-4e70-b167-de06c5dee9da None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:43.494927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c2ce4-2cc4-4e70-b167-de06c5dee9da None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:43.494927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c2ce4-2cc4-4e70-b167-de06c5dee9da None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:43.496376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:25:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:25:43.503755 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be981be2-a8cf-4e69-b1ba-80a846e08468 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:43.504558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be981be2-a8cf-4e69-b1ba-80a846e08468 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:43.504558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be981be2-a8cf-4e69-b1ba-80a846e08468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:43.580897 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:43.609813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-be981be2-a8cf-4e69-b1ba-80a846e08468 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:43.611204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-be981be2-a8cf-4e69-b1ba-80a846e08468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:43.735073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:43.891932 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c2ce4-2cc4-4e70-b167-de06c5dee9da None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:43.894753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-4b4c2ce4-2cc4-4e70-b167-de06c5dee9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:44.079034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c2ce4-2cc4-4e70-b167-de06c5dee9da None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:44.092860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:44.247527 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b3a08-5e04-4a82-9737-8dd46c9a8f4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:44.248581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b3a08-5e04-4a82-9737-8dd46c9a8f4f service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:44.249084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b3a08-5e04-4a82-9737-8dd46c9a8f4f service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:44.249575 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b3a08-5e04-4a82-9737-8dd46c9a8f4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:44.252831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-7d8b3a08-5e04-4a82-9737-8dd46c9a8f4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:44.365400 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b3a08-5e04-4a82-9737-8dd46c9a8f4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:44.366938 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b3a08-5e04-4a82-9737-8dd46c9a8f4f service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:44.389436 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 704/1422] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:45.632951 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21c265-d457-4d9a-bdbf-c5ef98397624 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:45.633789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21c265-d457-4d9a-bdbf-c5ef98397624 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:45.633789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21c265-d457-4d9a-bdbf-c5ef98397624 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:45.634959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:25:45.641623 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f377a-aa7a-4f71-aadc-7dc45d314ad0 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:45.642236 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f377a-aa7a-4f71-aadc-7dc45d314ad0 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:45.642236 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f377a-aa7a-4f71-aadc-7dc45d314ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:45.663353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:45.726021 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-8c8f377a-aa7a-4f71-aadc-7dc45d314ad0 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:45.727854 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-8c8f377a-aa7a-4f71-aadc-7dc45d314ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:45.871489 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:46.026381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21c265-d457-4d9a-bdbf-c5ef98397624 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:46.027949 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-2b21c265-d457-4d9a-bdbf-c5ef98397624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:46.166404 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:46.207489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21c265-d457-4d9a-bdbf-c5ef98397624 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:46.343424 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9d91b-6c93-4f34-a038-3f4d6aefb402 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:46.344209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9d91b-6c93-4f34-a038-3f4d6aefb402 service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:46.344374 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9d91b-6c93-4f34-a038-3f4d6aefb402 service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:46.344582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9d91b-6c93-4f34-a038-3f4d6aefb402 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:46.346265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-fcd9d91b-6c93-4f34-a038-3f4d6aefb402 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:46.447260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:46.462470 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9d91b-6c93-4f34-a038-3f4d6aefb402 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:46.464028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9d91b-6c93-4f34-a038-3f4d6aefb402 service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:46.486957 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 720/1425] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:47.840741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadcd5a-a906-4dd7-8da4-29407270b209 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:47.841621 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadcd5a-a906-4dd7-8da4-29407270b209 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:47.841621 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadcd5a-a906-4dd7-8da4-29407270b209 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:47.843333 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:25:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:25:47.851435 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f46f9-ef9c-484b-a570-7ac7d48f659f None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:47.856465 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f46f9-ef9c-484b-a570-7ac7d48f659f None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:47.856465 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f46f9-ef9c-484b-a570-7ac7d48f659f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:47.943126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-670f46f9-ef9c-484b-a570-7ac7d48f659f None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:47.944739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-670f46f9-ef9c-484b-a570-7ac7d48f659f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:48.072905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:48.233256 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcadcd5a-a906-4dd7-8da4-29407270b209 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:48.235966 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-bcadcd5a-a906-4dd7-8da4-29407270b209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:48.348905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:48.429377 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcadcd5a-a906-4dd7-8da4-29407270b209 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:48.568429 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1eaa8b-f24b-4f22-9ca3-6986549ffaf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:48.569059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eaa8b-f24b-4f22-9ca3-6986549ffaf2 service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:48.569433 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eaa8b-f24b-4f22-9ca3-6986549ffaf2 service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:48.569710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eaa8b-f24b-4f22-9ca3-6986549ffaf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:48.572042 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6c1eaa8b-f24b-4f22-9ca3-6986549ffaf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:48.643037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:48.690628 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1eaa8b-f24b-4f22-9ca3-6986549ffaf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:48.692472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1eaa8b-f24b-4f22-9ca3-6986549ffaf2 service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:48.716492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 707/1428] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:49.916727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c90031-604f-4916-9d2c-a3d1997dcf4c None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:49.917347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c90031-604f-4916-9d2c-a3d1997dcf4c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:49.917347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c90031-604f-4916-9d2c-a3d1997dcf4c None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:49.918706 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:25:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:25:49.926342 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d4f01-b834-44d5-9d19-4c110474e77c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:49.926924 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d4f01-b834-44d5-9d19-4c110474e77c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:49.926924 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d4f01-b834-44d5-9d19-4c110474e77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:50.023436 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-a54d4f01-b834-44d5-9d19-4c110474e77c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:50.024837 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a54d4f01-b834-44d5-9d19-4c110474e77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:50.148503 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:50.307065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c90031-604f-4916-9d2c-a3d1997dcf4c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:50.308677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-79c90031-604f-4916-9d2c-a3d1997dcf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:50.359192 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:50.474157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c90031-604f-4916-9d2c-a3d1997dcf4c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:50.611356 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700bacf-afd2-41c1-a230-b28c4f66979e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:50.612032 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700bacf-afd2-41c1-a230-b28c4f66979e service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:50.612324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700bacf-afd2-41c1-a230-b28c4f66979e service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:50.612752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700bacf-afd2-41c1-a230-b28c4f66979e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:50.614532 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9700bacf-afd2-41c1-a230-b28c4f66979e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:50.629892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:50.747414 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700bacf-afd2-41c1-a230-b28c4f66979e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:50.749068 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700bacf-afd2-41c1-a230-b28c4f66979e service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:50.775539 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 723/1431] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:25:51.963681 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d172-d521-44a7-b5fd-2ee3c44457c0 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:51.964520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d172-d521-44a7-b5fd-2ee3c44457c0 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:51.964520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d172-d521-44a7-b5fd-2ee3c44457c0 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:51.965593 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:25:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:25:51.975847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b111b-2335-4237-b59c-52cfb7ecd57e None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:51.976607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b111b-2335-4237-b59c-52cfb7ecd57e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:51.976742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b111b-2335-4237-b59c-52cfb7ecd57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:52.124018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-ab5b111b-2335-4237-b59c-52cfb7ecd57e None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:52.125343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ab5b111b-2335-4237-b59c-52cfb7ecd57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:52.203663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:52.239494 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:52.397988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400d172-d521-44a7-b5fd-2ee3c44457c0 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:52.400413 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-8400d172-d521-44a7-b5fd-2ee3c44457c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:52.420601 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:52.604306 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400d172-d521-44a7-b5fd-2ee3c44457c0 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:52.707933 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:52.738607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ebe05-c6f5-4d22-8ac1-6cb1d65969e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:52.739259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ebe05-c6f5-4d22-8ac1-6cb1d65969e6 service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:52.739544 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ebe05-c6f5-4d22-8ac1-6cb1d65969e6 service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:52.739822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ebe05-c6f5-4d22-8ac1-6cb1d65969e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:52.741518 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-bf4ebe05-c6f5-4d22-8ac1-6cb1d65969e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:52.882153 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ebe05-c6f5-4d22-8ac1-6cb1d65969e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:52.884112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ebe05-c6f5-4d22-8ac1-6cb1d65969e6 service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:52.912096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 710/1434] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:54.205641 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4189cab-0d27-40ed-b5b9-5a538727fe6e None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:54.206383 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4189cab-0d27-40ed-b5b9-5a538727fe6e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:54.206383 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4189cab-0d27-40ed-b5b9-5a538727fe6e None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:54.207684 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:25:54.216322 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ef5db-69e3-4316-8818-dc0022356bc3 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:54.217114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ef5db-69e3-4316-8818-dc0022356bc3 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:54.217114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ef5db-69e3-4316-8818-dc0022356bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:54.330526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-5c1ef5db-69e3-4316-8818-dc0022356bc3 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:54.332028 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5c1ef5db-69e3-4316-8818-dc0022356bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:54.384250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:54.480618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:25:54.651023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4189cab-0d27-40ed-b5b9-5a538727fe6e None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:54.653734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c4189cab-0d27-40ed-b5b9-5a538727fe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:54.875279 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4189cab-0d27-40ed-b5b9-5a538727fe6e None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:54.957139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:55.036370 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337c14d-b38c-4379-b037-7e84d5f33ce1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:55.036987 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337c14d-b38c-4379-b037-7e84d5f33ce1 service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:55.037268 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337c14d-b38c-4379-b037-7e84d5f33ce1 service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:55.037549 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337c14d-b38c-4379-b037-7e84d5f33ce1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:55.039704 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1337c14d-b38c-4379-b037-7e84d5f33ce1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:55.167786 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337c14d-b38c-4379-b037-7e84d5f33ce1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:55.169439 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337c14d-b38c-4379-b037-7e84d5f33ce1 service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:55.192464 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 726/1437] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:25:56.524510 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe14c4-22fb-45dd-9eaf-5033cf69886e None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:56.525408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe14c4-22fb-45dd-9eaf-5033cf69886e None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:56.526322 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe14c4-22fb-45dd-9eaf-5033cf69886e None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:56.528415 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:25:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:25:56.538406 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3771ca-cd62-429f-8a20-df64d5f06764 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:56.539268 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3771ca-cd62-429f-8a20-df64d5f06764 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:56.540142 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3771ca-cd62-429f-8a20-df64d5f06764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:56.784151 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-5a3771ca-cd62-429f-8a20-df64d5f06764 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:25:56.785417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-5a3771ca-cd62-429f-8a20-df64d5f06764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:56.788006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:56.919347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:25:57.226264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe14c4-22fb-45dd-9eaf-5033cf69886e None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:25:57.229413 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-abfe14c4-22fb-45dd-9eaf-5033cf69886e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:57.489645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe14c4-22fb-45dd-9eaf-5033cf69886e None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:25:57.535888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:25:57.665146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb5d5b-7332-4e37-a9c2-41bf293f509e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:25:57.668700 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb5d5b-7332-4e37-a9c2-41bf293f509e service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:25:57.669255 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb5d5b-7332-4e37-a9c2-41bf293f509e service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:25:57.669742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb5d5b-7332-4e37-a9c2-41bf293f509e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:25:57.675803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ccdb5d5b-7332-4e37-a9c2-41bf293f509e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:25:57.830740 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb5d5b-7332-4e37-a9c2-41bf293f509e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:25:57.833591 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb5d5b-7332-4e37-a9c2-41bf293f509e service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:25:57.870116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 713/1440] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:25:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:00.490613 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db5e35-814e-42d7-828e-f847dacef5c4 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:00.491180 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db5e35-814e-42d7-828e-f847dacef5c4 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:00.491645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db5e35-814e-42d7-828e-f847dacef5c4 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:00.494710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:26:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:26:00.502926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2be24-03db-4e49-b174-93894b0890fa None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:00.503572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2be24-03db-4e49-b174-93894b0890fa None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:00.503572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2be24-03db-4e49-b174-93894b0890fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:00.558794 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:00.591290 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-80a2be24-03db-4e49-b174-93894b0890fa None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:00.592579 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-80a2be24-03db-4e49-b174-93894b0890fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:00.725657 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:01.027752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db5e35-814e-42d7-828e-f847dacef5c4 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:01.029284 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-94db5e35-814e-42d7-828e-f847dacef5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:01.161686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db5e35-814e-42d7-828e-f847dacef5c4 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:01.212577 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:01.257060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef493198-4609-403d-a418-6e281faa156d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:01.257675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef493198-4609-403d-a418-6e281faa156d service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:01.257951 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef493198-4609-403d-a418-6e281faa156d service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:01.258768 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef493198-4609-403d-a418-6e281faa156d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:01.260595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ef493198-4609-403d-a418-6e281faa156d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:01.385158 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef493198-4609-403d-a418-6e281faa156d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:01.386583 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef493198-4609-403d-a418-6e281faa156d service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:01.405827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 729/1443] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:03.964000 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e600ac0-def0-4692-9058-147c5cab5193 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:03.964622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e600ac0-def0-4692-9058-147c5cab5193 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:03.964622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e600ac0-def0-4692-9058-147c5cab5193 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:03.965974 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:26:03.971759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142953d-7096-44c7-9166-3a96c2d16a0e None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:03.983514 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142953d-7096-44c7-9166-3a96c2d16a0e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:03.983514 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142953d-7096-44c7-9166-3a96c2d16a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:04.052520 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-f142953d-7096-44c7-9166-3a96c2d16a0e None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:04.053761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f142953d-7096-44c7-9166-3a96c2d16a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:04.070385 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:04.161748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:04.449777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e600ac0-def0-4692-9058-147c5cab5193 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:04.451469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-3e600ac0-def0-4692-9058-147c5cab5193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:04.585225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e600ac0-def0-4692-9058-147c5cab5193 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:04.599668 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:04.810797 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0669bf-821f-4f5e-abef-7a80752b56cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:04.811860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0669bf-821f-4f5e-abef-7a80752b56cd service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:04.812357 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0669bf-821f-4f5e-abef-7a80752b56cd service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:04.812847 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0669bf-821f-4f5e-abef-7a80752b56cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:04.816066 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9e0669bf-821f-4f5e-abef-7a80752b56cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:04.990171 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0669bf-821f-4f5e-abef-7a80752b56cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:04.991875 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0669bf-821f-4f5e-abef-7a80752b56cd service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:05.025490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 716/1446] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:08.000080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900840fc-672b-4d1a-8279-200348c194a1 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:08.000685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900840fc-672b-4d1a-8279-200348c194a1 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:08.000685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900840fc-672b-4d1a-8279-200348c194a1 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:08.002006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:26:08.010625 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbd5f7-0829-4fa1-993a-bb2d8f398e35 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:08.011352 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbd5f7-0829-4fa1-993a-bb2d8f398e35 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:08.011352 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbd5f7-0829-4fa1-993a-bb2d8f398e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:08.070860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:08.093555 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-68dbd5f7-0829-4fa1-993a-bb2d8f398e35 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:08.094905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-68dbd5f7-0829-4fa1-993a-bb2d8f398e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:08.206386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:08.503267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900840fc-672b-4d1a-8279-200348c194a1 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:08.504842 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-900840fc-672b-4d1a-8279-200348c194a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:08.627970 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900840fc-672b-4d1a-8279-200348c194a1 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:08.672026 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:08.712465 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d347dd2-348d-471b-9fa9-7f6a762d9d67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:08.713103 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d347dd2-348d-471b-9fa9-7f6a762d9d67 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:08.713378 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d347dd2-348d-471b-9fa9-7f6a762d9d67 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:08.713656 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d347dd2-348d-471b-9fa9-7f6a762d9d67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:08.715410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8d347dd2-348d-471b-9fa9-7f6a762d9d67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:08.853387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d347dd2-348d-471b-9fa9-7f6a762d9d67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:08.855050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d347dd2-348d-471b-9fa9-7f6a762d9d67 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:08.875510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 732/1449] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:11.421831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e871d-8411-4ded-b3ce-e89711bda13b None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:11.422438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e871d-8411-4ded-b3ce-e89711bda13b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:11.422438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e871d-8411-4ded-b3ce-e89711bda13b None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:11.423720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:26:11.430599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d1592-5689-4a69-a0ab-baa6ffe42b84 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:11.431183 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d1592-5689-4a69-a0ab-baa6ffe42b84 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:11.431451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d1592-5689-4a69-a0ab-baa6ffe42b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:11.515819 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-137d1592-5689-4a69-a0ab-baa6ffe42b84 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:11.517164 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-137d1592-5689-4a69-a0ab-baa6ffe42b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:11.548935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:11.640981 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:11.946937 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e871d-8411-4ded-b3ce-e89711bda13b None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:11.948779 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-df1e871d-8411-4ded-b3ce-e89711bda13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:12.065192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e871d-8411-4ded-b3ce-e89711bda13b None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:12.092395 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:12.152181 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646e864-7222-4f30-a666-a8bd6b5595a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:12.152795 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646e864-7222-4f30-a666-a8bd6b5595a2 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:12.153071 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646e864-7222-4f30-a666-a8bd6b5595a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:12.153342 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646e864-7222-4f30-a666-a8bd6b5595a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:12.155153 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-1646e864-7222-4f30-a666-a8bd6b5595a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:12.296823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646e864-7222-4f30-a666-a8bd6b5595a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:12.299706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646e864-7222-4f30-a666-a8bd6b5595a2 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:12.332043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 719/1452] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:14.730030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7226d16-6da5-44b3-8d84-57354340a336 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:14.730618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7226d16-6da5-44b3-8d84-57354340a336 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:14.730618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7226d16-6da5-44b3-8d84-57354340a336 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:14.732171 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:26:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:26:14.738082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3eb224-76e2-48ee-9c75-1a7f902fa782 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:14.738602 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3eb224-76e2-48ee-9c75-1a7f902fa782 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:14.738602 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3eb224-76e2-48ee-9c75-1a7f902fa782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:14.818610 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:14.822961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-eb3eb224-76e2-48ee-9c75-1a7f902fa782 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:14.824395 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-eb3eb224-76e2-48ee-9c75-1a7f902fa782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:14.959281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:15.272288 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7226d16-6da5-44b3-8d84-57354340a336 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:15.280827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b7226d16-6da5-44b3-8d84-57354340a336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:15.393522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7226d16-6da5-44b3-8d84-57354340a336 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:15.447211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:15.474266 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc52ce-3914-414a-99db-656950794fc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:15.475123 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc52ce-3914-414a-99db-656950794fc5 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:15.475523 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc52ce-3914-414a-99db-656950794fc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:15.476153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc52ce-3914-414a-99db-656950794fc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:15.478733 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-59fc52ce-3914-414a-99db-656950794fc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:15.593967 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc52ce-3914-414a-99db-656950794fc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:15.595457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc52ce-3914-414a-99db-656950794fc5 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:15.615488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 735/1455] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:26:18.014349 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acd16c-6352-4afb-afca-d158c18d3f09 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:18.014956 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acd16c-6352-4afb-afca-d158c18d3f09 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:18.014956 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acd16c-6352-4afb-afca-d158c18d3f09 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:18.016244 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:26:18.022036 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e86d3-1904-48dc-853a-02fd85986dcc None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:18.022567 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e86d3-1904-48dc-853a-02fd85986dcc None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:18.022567 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e86d3-1904-48dc-853a-02fd85986dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:18.108240 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-a91e86d3-1904-48dc-853a-02fd85986dcc None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:18.109467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a91e86d3-1904-48dc-853a-02fd85986dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:18.144521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:18.229502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:18.526153 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acd16c-6352-4afb-afca-d158c18d3f09 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:18.527720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f8acd16c-6352-4afb-afca-d158c18d3f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:18.638362 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acd16c-6352-4afb-afca-d158c18d3f09 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:18.667556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:18.717035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066f136-7bf3-4320-8ae8-14175747f9a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:18.717653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066f136-7bf3-4320-8ae8-14175747f9a4 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:18.717936 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066f136-7bf3-4320-8ae8-14175747f9a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:18.718238 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066f136-7bf3-4320-8ae8-14175747f9a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:18.719928 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a066f136-7bf3-4320-8ae8-14175747f9a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:18.845385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066f136-7bf3-4320-8ae8-14175747f9a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:18.850744 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066f136-7bf3-4320-8ae8-14175747f9a4 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:18.871729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 722/1458] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:21.202374 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d095d-01f6-482f-ba7f-40f94901bc75 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:21.202913 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d095d-01f6-482f-ba7f-40f94901bc75 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:21.202913 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d095d-01f6-482f-ba7f-40f94901bc75 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:21.204295 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:26:21.209947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a54a8c-e7f5-4a88-b89f-5986bcb497aa None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:21.210414 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a54a8c-e7f5-4a88-b89f-5986bcb497aa None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:21.210531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a54a8c-e7f5-4a88-b89f-5986bcb497aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:21.279531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-a7a54a8c-e7f5-4a88-b89f-5986bcb497aa None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:21.280758 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a7a54a8c-e7f5-4a88-b89f-5986bcb497aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:21.290088 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:21.381968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:21.670432 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d095d-01f6-482f-ba7f-40f94901bc75 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:21.672012 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-458d095d-01f6-482f-ba7f-40f94901bc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:21.786168 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d095d-01f6-482f-ba7f-40f94901bc75 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:21.842191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:21.861216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d1f5b-ee25-4092-919d-b76a8f1ed3ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:21.861862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d1f5b-ee25-4092-919d-b76a8f1ed3ce service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:21.862224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d1f5b-ee25-4092-919d-b76a8f1ed3ce service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:21.862666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d1f5b-ee25-4092-919d-b76a8f1ed3ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:21.864357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e97d1f5b-ee25-4092-919d-b76a8f1ed3ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:21.991368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d1f5b-ee25-4092-919d-b76a8f1ed3ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:21.994540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d1f5b-ee25-4092-919d-b76a8f1ed3ce service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:22.016657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 738/1461] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:24.438600 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378ac33-2c09-4a13-842c-1990772bb5df None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:24.439549 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378ac33-2c09-4a13-842c-1990772bb5df None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:24.439549 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378ac33-2c09-4a13-842c-1990772bb5df None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:24.440826 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:26:24.451574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1517b7-f289-4f69-a85e-7aca1b80e457 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:24.452184 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1517b7-f289-4f69-a85e-7aca1b80e457 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:24.452184 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1517b7-f289-4f69-a85e-7aca1b80e457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:24.540615 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-bb1517b7-f289-4f69-a85e-7aca1b80e457 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:24.541935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-bb1517b7-f289-4f69-a85e-7aca1b80e457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:24.599174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:24.674465 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:24.970785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378ac33-2c09-4a13-842c-1990772bb5df None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:24.972358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9378ac33-2c09-4a13-842c-1990772bb5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:25.081381 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378ac33-2c09-4a13-842c-1990772bb5df None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:25.123151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:25.162626 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dbf7c-21cb-463d-86ca-26493b3fe797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:25.163275 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dbf7c-21cb-463d-86ca-26493b3fe797 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:25.163557 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dbf7c-21cb-463d-86ca-26493b3fe797 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:25.163840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dbf7c-21cb-463d-86ca-26493b3fe797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:25.167429 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-7a7dbf7c-21cb-463d-86ca-26493b3fe797 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:25.291739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7dbf7c-21cb-463d-86ca-26493b3fe797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:25.293228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7dbf7c-21cb-463d-86ca-26493b3fe797 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:25.314098 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 725/1464] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:27.665096 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da91a2-5a1e-41f9-ac03-eba4079f4e13 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:27.665669 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da91a2-5a1e-41f9-ac03-eba4079f4e13 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:27.665669 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da91a2-5a1e-41f9-ac03-eba4079f4e13 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:27.667019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:26:27.672925 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5b15d-3e65-4382-b947-d654e3d937be None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:27.673426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5b15d-3e65-4382-b947-d654e3d937be None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:27.673530 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5b15d-3e65-4382-b947-d654e3d937be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:27.749136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-00d5b15d-3e65-4382-b947-d654e3d937be None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:27.750274 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-00d5b15d-3e65-4382-b947-d654e3d937be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:27.770284 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:27.857565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:28.148738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da91a2-5a1e-41f9-ac03-eba4079f4e13 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:28.150327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-02da91a2-5a1e-41f9-ac03-eba4079f4e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:28.268308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da91a2-5a1e-41f9-ac03-eba4079f4e13 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:28.342337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:28.349368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b97c08-8e07-4aed-897a-6f47d5f2b12c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:28.350028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b97c08-8e07-4aed-897a-6f47d5f2b12c service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:28.350307 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b97c08-8e07-4aed-897a-6f47d5f2b12c service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:28.350574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b97c08-8e07-4aed-897a-6f47d5f2b12c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:28.352906 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-44b97c08-8e07-4aed-897a-6f47d5f2b12c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:28.472812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b97c08-8e07-4aed-897a-6f47d5f2b12c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:28.474247 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b97c08-8e07-4aed-897a-6f47d5f2b12c service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:28.494945 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 741/1467] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:30.824130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72c570-13ac-44ea-a7c3-22eb846511ef None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:30.824728 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72c570-13ac-44ea-a7c3-22eb846511ef None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:30.824728 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72c570-13ac-44ea-a7c3-22eb846511ef None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:30.826056 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:26:30.832970 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653ee20-0433-4dd0-8591-eee9aa39a229 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:30.833535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653ee20-0433-4dd0-8591-eee9aa39a229 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:30.833640 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653ee20-0433-4dd0-8591-eee9aa39a229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:30.906451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-d653ee20-0433-4dd0-8591-eee9aa39a229 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:30.907742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d653ee20-0433-4dd0-8591-eee9aa39a229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:30.958657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:31.014029 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:31.315695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72c570-13ac-44ea-a7c3-22eb846511ef None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:31.317240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2a72c570-13ac-44ea-a7c3-22eb846511ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:31.424268 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72c570-13ac-44ea-a7c3-22eb846511ef None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:31.473883 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:31.501950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71393ae0-56fc-49dd-930a-d34b6a23ebd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:31.502682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71393ae0-56fc-49dd-930a-d34b6a23ebd9 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:31.502987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71393ae0-56fc-49dd-930a-d34b6a23ebd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:31.503479 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71393ae0-56fc-49dd-930a-d34b6a23ebd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:31.505704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-71393ae0-56fc-49dd-930a-d34b6a23ebd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:31.627161 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71393ae0-56fc-49dd-930a-d34b6a23ebd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:31.628617 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71393ae0-56fc-49dd-930a-d34b6a23ebd9 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:31.650767 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 728/1470] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:41.012183 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=22 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:41.647712 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=66 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:53.230958 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5160f5-99fa-4a02-acd6-9baf3281cfd3 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:53.232404 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5160f5-99fa-4a02-acd6-9baf3281cfd3 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:53.232404 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5160f5-99fa-4a02-acd6-9baf3281cfd3 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:53.233020 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 743/1471] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:26:53.239488 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58a18a-5fcb-4cc0-84df-6c581f276153 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:53.240141 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58a18a-5fcb-4cc0-84df-6c581f276153 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:53.240141 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58a18a-5fcb-4cc0-84df-6c581f276153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:53.325211 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-5a58a18a-5fcb-4cc0-84df-6c581f276153 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:53.326452 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5a58a18a-5fcb-4cc0-84df-6c581f276153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:53.438006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 729/1472] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:53.759251 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97aeafc-53a2-4909-8865-2d6fabecc4d2 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:53.759474 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97aeafc-53a2-4909-8865-2d6fabecc4d2 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:53.759847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97aeafc-53a2-4909-8865-2d6fabecc4d2 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:53.761218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 744/1473] 10.222.0.62 () {54 vars in 961 bytes} [Wed Sep 7 13:26:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:26:53.766581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7928d7c-8a27-436f-bad4-1014170753df None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:53.766861 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7928d7c-8a27-436f-bad4-1014170753df None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:53.767158 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7928d7c-8a27-436f-bad4-1014170753df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:53.860103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c7928d7c-8a27-436f-bad4-1014170753df None None] MFA Rules not processed for user `d1a48805d95640489d7ce3333dd687b9`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:53.861316 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c7928d7c-8a27-436f-bad4-1014170753df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:53.884069 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:53.957769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 730/1474] 10.222.0.62 () {58 vars in 1058 bytes} [Wed Sep 7 13:26:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:53.967831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97aeafc-53a2-4909-8865-2d6fabecc4d2 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:53.969446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a97aeafc-53a2-4909-8865-2d6fabecc4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:54.095299 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97aeafc-53a2-4909-8865-2d6fabecc4d2 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:54.181827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc428be7-3154-41bb-9431-f06229e145e0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1a48805d95640489d7ce3333dd687b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:54.182435 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc428be7-3154-41bb-9431-f06229e145e0 service glance] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:54.182718 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc428be7-3154-41bb-9431-f06229e145e0 service glance] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:54.183002 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc428be7-3154-41bb-9431-f06229e145e0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:54.184670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-fc428be7-3154-41bb-9431-f06229e145e0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:54.243084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:54.305214 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc428be7-3154-41bb-9431-f06229e145e0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:54.306675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc428be7-3154-41bb-9431-f06229e145e0 service glance] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:54.326534 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 745/1475] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:26:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:26:56.704941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777a1b4-a60f-4efb-91f2-44d4abdf93b3 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:56.705581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777a1b4-a60f-4efb-91f2-44d4abdf93b3 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:56.705581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777a1b4-a60f-4efb-91f2-44d4abdf93b3 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:56.706757 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 731/1476] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:26:56.712623 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157683a-7d96-42b0-bd81-2a24c3da40f2 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:56.713183 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157683a-7d96-42b0-bd81-2a24c3da40f2 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:56.713351 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157683a-7d96-42b0-bd81-2a24c3da40f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:56.797861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-6157683a-7d96-42b0-bd81-2a24c3da40f2 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:56.799374 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-6157683a-7d96-42b0-bd81-2a24c3da40f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:56.911614 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 746/1477] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:26:57.194553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777a1b4-a60f-4efb-91f2-44d4abdf93b3 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:26:57.196356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9777a1b4-a60f-4efb-91f2-44d4abdf93b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:26:57.364014 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777a1b4-a60f-4efb-91f2-44d4abdf93b3 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:26:57.448979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:26:57.499343 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5ff91-1543-460e-ad20-94f4ae2843a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:26:57.499953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5ff91-1543-460e-ad20-94f4ae2843a0 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:57.500227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5ff91-1543-460e-ad20-94f4ae2843a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:57.500524 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5ff91-1543-460e-ad20-94f4ae2843a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:57.501157 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5ff91-1543-460e-ad20-94f4ae2843a0 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:26:57.502896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: WARNING py.warnings [None req-02b5ff91-1543-460e-ad20-94f4ae2843a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:26:57.502896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: warnings.warn(msg) Sep 07 13:26:57.502896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: Sep 07 13:26:57.504047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5ff91-1543-460e-ad20-94f4ae2843a0 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:26:57.524677 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 732/1478] 10.222.0.62 () {56 vars in 1108 bytes} [Wed Sep 7 13:26:57 2022] GET /identity/v3/projects => generated 2553 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:26:59.799045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c1bd6-5eeb-4c2d-be0e-6d91ef5348fd None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:59.799779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c1bd6-5eeb-4c2d-be0e-6d91ef5348fd None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:59.799779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c1bd6-5eeb-4c2d-be0e-6d91ef5348fd None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:59.801370 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 747/1479] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:26:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:26:59.809040 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eedb93-6343-4990-88f4-4a3742144320 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:26:59.809699 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eedb93-6343-4990-88f4-4a3742144320 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:26:59.809699 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eedb93-6343-4990-88f4-4a3742144320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:26:59.908710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-a5eedb93-6343-4990-88f4-4a3742144320 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:26:59.910578 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a5eedb93-6343-4990-88f4-4a3742144320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:00.039355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 733/1480] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:27:00.376521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c1bd6-5eeb-4c2d-be0e-6d91ef5348fd None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:00.382458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d16c1bd6-5eeb-4c2d-be0e-6d91ef5348fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:00.395331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:00.542726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c1bd6-5eeb-4c2d-be0e-6d91ef5348fd None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:00.634002 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:00.680175 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c75437-412a-4b0e-9cd9-6b74f3ab3b71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:00.681054 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c75437-412a-4b0e-9cd9-6b74f3ab3b71 service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:00.681529 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c75437-412a-4b0e-9cd9-6b74f3ab3b71 service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:00.681897 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c75437-412a-4b0e-9cd9-6b74f3ab3b71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:00.683744 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-82c75437-412a-4b0e-9cd9-6b74f3ab3b71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:00.804203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c75437-412a-4b0e-9cd9-6b74f3ab3b71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:00.805858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c75437-412a-4b0e-9cd9-6b74f3ab3b71 service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:00.827673 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 748/1481] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:03.255281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17404c8a-d22f-4f03-867f-0681edf50161 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:03.255845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17404c8a-d22f-4f03-867f-0681edf50161 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:03.255845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17404c8a-d22f-4f03-867f-0681edf50161 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:03.257271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 734/1482] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:27:03.263277 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0668476-7d72-4eaf-ab14-087e8b5ae45f None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:03.263807 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0668476-7d72-4eaf-ab14-087e8b5ae45f None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:03.263807 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0668476-7d72-4eaf-ab14-087e8b5ae45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:03.332238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-a0668476-7d72-4eaf-ab14-087e8b5ae45f None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:27:03.333468 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a0668476-7d72-4eaf-ab14-087e8b5ae45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:03.381290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:03.435560 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 749/1483] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:27:03.728231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17404c8a-d22f-4f03-867f-0681edf50161 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:03.729807 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-17404c8a-d22f-4f03-867f-0681edf50161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:03.753291 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:03.840009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17404c8a-d22f-4f03-867f-0681edf50161 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:03.917368 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f5820-ff99-4bbb-b2cb-5bd15866f4b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:03.917979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f5820-ff99-4bbb-b2cb-5bd15866f4b6 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:03.918253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f5820-ff99-4bbb-b2cb-5bd15866f4b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:03.918522 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f5820-ff99-4bbb-b2cb-5bd15866f4b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:03.920197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-3b5f5820-ff99-4bbb-b2cb-5bd15866f4b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:04.002661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:04.038593 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f5820-ff99-4bbb-b2cb-5bd15866f4b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:04.040073 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f5820-ff99-4bbb-b2cb-5bd15866f4b6 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:04.061225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 735/1484] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:06.390564 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58596d-040f-4104-9a4b-7b6030acc8ff None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:06.391238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58596d-040f-4104-9a4b-7b6030acc8ff None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:06.391238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f58596d-040f-4104-9a4b-7b6030acc8ff None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:06.392517 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 750/1485] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:27:06.400478 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf993afd-001a-4cec-bdda-0d3554dee4d2 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:06.400990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf993afd-001a-4cec-bdda-0d3554dee4d2 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:06.401099 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf993afd-001a-4cec-bdda-0d3554dee4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:06.469086 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-cf993afd-001a-4cec-bdda-0d3554dee4d2 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:27:06.470362 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-cf993afd-001a-4cec-bdda-0d3554dee4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:06.571165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 736/1486] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:27:06.870210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58596d-040f-4104-9a4b-7b6030acc8ff None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:06.871822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-3f58596d-040f-4104-9a4b-7b6030acc8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:06.991033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f58596d-040f-4104-9a4b-7b6030acc8ff None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:07.067533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:07.069332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f0782-0d92-4daf-86fd-4539e7777eef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:07.070486 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f0782-0d92-4daf-86fd-4539e7777eef service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:07.070733 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f0782-0d92-4daf-86fd-4539e7777eef service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:07.071055 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f0782-0d92-4daf-86fd-4539e7777eef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:07.072646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d38f0782-0d92-4daf-86fd-4539e7777eef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:07.211347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f0782-0d92-4daf-86fd-4539e7777eef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:07.212919 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f0782-0d92-4daf-86fd-4539e7777eef service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:07.233494 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 751/1487] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:09.659914 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5295a-5f86-4d37-b485-05facd466b60 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:09.660480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5295a-5f86-4d37-b485-05facd466b60 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:09.660480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5295a-5f86-4d37-b485-05facd466b60 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:09.661917 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 737/1488] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:27:09.667749 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a2a2c-a544-4dd2-ba2b-e4c474217142 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:09.668513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a2a2c-a544-4dd2-ba2b-e4c474217142 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:09.668686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a2a2c-a544-4dd2-ba2b-e4c474217142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:09.738724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b66a2a2c-a544-4dd2-ba2b-e4c474217142 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:27:09.739904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b66a2a2c-a544-4dd2-ba2b-e4c474217142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:09.796074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:09.844171 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 752/1489] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:27:10.142335 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5295a-5f86-4d37-b485-05facd466b60 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:10.143949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-56c5295a-5f86-4d37-b485-05facd466b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:10.182267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:10.262814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5295a-5f86-4d37-b485-05facd466b60 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:10.349074 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d8af9-e401-4a62-9ec5-fe7ab632e6f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:10.349687 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d8af9-e401-4a62-9ec5-fe7ab632e6f7 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:10.350097 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d8af9-e401-4a62-9ec5-fe7ab632e6f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:10.350271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d8af9-e401-4a62-9ec5-fe7ab632e6f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:10.353085 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6f9d8af9-e401-4a62-9ec5-fe7ab632e6f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:10.442991 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:10.478178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d8af9-e401-4a62-9ec5-fe7ab632e6f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:10.479665 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d8af9-e401-4a62-9ec5-fe7ab632e6f7 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:10.498393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 738/1490] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:13.276260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5abef-adfd-498b-a369-733189215e15 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:13.277098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5abef-adfd-498b-a369-733189215e15 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:13.277098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5abef-adfd-498b-a369-733189215e15 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:13.278824 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 753/1491] 10.222.0.62 () {54 vars in 927 bytes} [Wed Sep 7 13:27:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:27:13.284717 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4bd79-ce0a-4550-bc01-fac7f5b55941 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:13.285264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4bd79-ce0a-4550-bc01-fac7f5b55941 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:13.285264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4bd79-ce0a-4550-bc01-fac7f5b55941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:13.357036 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-68c4bd79-ce0a-4550-bc01-fac7f5b55941 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:27:13.358287 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-68c4bd79-ce0a-4550-bc01-fac7f5b55941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:13.459562 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 739/1492] 10.222.0.62 () {58 vars in 1024 bytes} [Wed Sep 7 13:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:27:13.754561 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5abef-adfd-498b-a369-733189215e15 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:27:13.756184 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-12f5abef-adfd-498b-a369-733189215e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:13.901127 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5abef-adfd-498b-a369-733189215e15 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:27:14.003263 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c322aa4-9dc2-4279-8b60-d6d6658cab4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:27:14.003810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c322aa4-9dc2-4279-8b60-d6d6658cab4e service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:27:14.004093 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c322aa4-9dc2-4279-8b60-d6d6658cab4e service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:27:14.004438 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c322aa4-9dc2-4279-8b60-d6d6658cab4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:27:14.006182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1c322aa4-9dc2-4279-8b60-d6d6658cab4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:27:14.020189 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:14.152083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c322aa4-9dc2-4279-8b60-d6d6658cab4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:27:14.153591 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c322aa4-9dc2-4279-8b60-d6d6658cab4e service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:27:14.174218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 754/1493] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:27:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:27:23.457694 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=82 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:27:24.171383 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=50 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:28:43.497211 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ee2ba-ef2e-4c43-b5a1-f06e21b1a7a9 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:43.497773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ee2ba-ef2e-4c43-b5a1-f06e21b1a7a9 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:43.497773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ee2ba-ef2e-4c43-b5a1-f06e21b1a7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:43.585665 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-5f2ee2ba-ef2e-4c43-b5a1-f06e21b1a7a9 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:28:43.587209 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5f2ee2ba-ef2e-4c43-b5a1-f06e21b1a7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:28:43.698949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 740/1494] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:28:43.707084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc7ccc-71c4-40b8-8a13-d1e3ec9cdb2a None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:43.707886 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc7ccc-71c4-40b8-8a13-d1e3ec9cdb2a None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:43.707886 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc7ccc-71c4-40b8-8a13-d1e3ec9cdb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:43.729039 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b5bc7ccc-71c4-40b8-8a13-d1e3ec9cdb2a None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:28:43.730284 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b5bc7ccc-71c4-40b8-8a13-d1e3ec9cdb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:28:43.739178 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 755/1495] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:28:43.744583 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ee2ba-ef2e-4c43-b5a1-f06e21b1a7a9 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:28:43.754442 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ee2ba-ef2e-4c43-b5a1-f06e21b1a7a9 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:28:43.760160 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59db86c-3867-446a-898f-dfc1a3de953d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:28:43.760775 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59db86c-3867-446a-898f-dfc1a3de953d admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:43.761060 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59db86c-3867-446a-898f-dfc1a3de953d admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:43.761357 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59db86c-3867-446a-898f-dfc1a3de953d admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:43.762021 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59db86c-3867-446a-898f-dfc1a3de953d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:28:43.762323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59db86c-3867-446a-898f-dfc1a3de953d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:28:43.763376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: WARNING py.warnings [None req-d59db86c-3867-446a-898f-dfc1a3de953d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:28:43.763376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: warnings.warn(msg) Sep 07 13:28:43.763376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: Sep 07 13:28:43.764267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59db86c-3867-446a-898f-dfc1a3de953d admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:28:43.777053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 741/1496] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:28:43.785410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc7ccc-71c4-40b8-8a13-d1e3ec9cdb2a None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:28:43.795676 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc7ccc-71c4-40b8-8a13-d1e3ec9cdb2a None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:28:43.800691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7547c-898c-468a-94ec-4713fcb3b1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:28:43.801306 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7547c-898c-468a-94ec-4713fcb3b1d0 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:43.801582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7547c-898c-468a-94ec-4713fcb3b1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:43.801854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7547c-898c-468a-94ec-4713fcb3b1d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:43.802507 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7547c-898c-468a-94ec-4713fcb3b1d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:28:43.804375 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: WARNING py.warnings [None req-4df7547c-898c-468a-94ec-4713fcb3b1d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:28:43.804375 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: warnings.warn(msg) Sep 07 13:28:43.804375 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: Sep 07 13:28:43.805566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7547c-898c-468a-94ec-4713fcb3b1d0 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:28:43.907066 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 756/1497] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:28:43 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:28:43.911372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59db86c-3867-446a-898f-dfc1a3de953d admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:28:43.922026 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59db86c-3867-446a-898f-dfc1a3de953d admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:28:43.975330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:28:43.982620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e6a0c-8308-43e1-a031-97a00ad93327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:28:43.983294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e6a0c-8308-43e1-a031-97a00ad93327 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:43.983578 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e6a0c-8308-43e1-a031-97a00ad93327 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:43.983851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e6a0c-8308-43e1-a031-97a00ad93327 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:43.984503 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e6a0c-8308-43e1-a031-97a00ad93327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:28:43.985519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: WARNING py.warnings [None req-9a5e6a0c-8308-43e1-a031-97a00ad93327 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:28:43.985519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: warnings.warn(msg) Sep 07 13:28:43.985519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: Sep 07 13:28:43.986775 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e6a0c-8308-43e1-a031-97a00ad93327 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:28:44.102419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 742/1498] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:28:43 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:28:44.106706 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7547c-898c-468a-94ec-4713fcb3b1d0 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:28:44.118472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7547c-898c-468a-94ec-4713fcb3b1d0 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:28:44.123185 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9cd1f-7c98-4160-b45a-ba6e102eb73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:28:44.123796 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9cd1f-7c98-4160-b45a-ba6e102eb73e admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:44.124074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9cd1f-7c98-4160-b45a-ba6e102eb73e admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:44.124346 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9cd1f-7c98-4160-b45a-ba6e102eb73e admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:44.124988 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9cd1f-7c98-4160-b45a-ba6e102eb73e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:28:44.126043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: WARNING py.warnings [None req-bdc9cd1f-7c98-4160-b45a-ba6e102eb73e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:28:44.126043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: warnings.warn(msg) Sep 07 13:28:44.126043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: Sep 07 13:28:44.126920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9cd1f-7c98-4160-b45a-ba6e102eb73e admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:28:44.141339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 757/1499] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:28:44.145498 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e6a0c-8308-43e1-a031-97a00ad93327 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:28:44.155801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e6a0c-8308-43e1-a031-97a00ad93327 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:28:44.160449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a34eba-e23b-4f5e-8488-82826378ebf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:28:44.161110 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a34eba-e23b-4f5e-8488-82826378ebf4 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:44.161389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a34eba-e23b-4f5e-8488-82826378ebf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:44.161816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a34eba-e23b-4f5e-8488-82826378ebf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:44.162464 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a34eba-e23b-4f5e-8488-82826378ebf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:28:44.163527 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: WARNING py.warnings [None req-82a34eba-e23b-4f5e-8488-82826378ebf4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:28:44.163527 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: warnings.warn(msg) Sep 07 13:28:44.163527 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: Sep 07 13:28:44.164394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a34eba-e23b-4f5e-8488-82826378ebf4 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:28:44.179067 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 743/1500] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:28:44.183738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9cd1f-7c98-4160-b45a-ba6e102eb73e admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:28:44.194870 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9cd1f-7c98-4160-b45a-ba6e102eb73e admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:28:44.199598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866343a1-6c79-447a-b075-7d5eb474e6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:28:44.200246 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866343a1-6c79-447a-b075-7d5eb474e6c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:44.200520 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866343a1-6c79-447a-b075-7d5eb474e6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:44.200847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866343a1-6c79-447a-b075-7d5eb474e6c4 admin admin] PATH_INFO: `/v3/projects/e803464281e645d4b5aada8cd690716a/users/0670d87cb9284368ad13959c5554b4be/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:44.216626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866343a1-6c79-447a-b075-7d5eb474e6c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e803464281e645d4b5aada8cd690716a, user_id=0670d87cb9284368ad13959c5554b4be, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:28:44.217537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: WARNING py.warnings [None req-866343a1-6c79-447a-b075-7d5eb474e6c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:28:44.217537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: warnings.warn(msg) Sep 07 13:28:44.217537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: Sep 07 13:28:44.219480 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866343a1-6c79-447a-b075-7d5eb474e6c4 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:28:44.297135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 758/1501] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:28:44 2022] PUT /identity/v3/projects/e803464281e645d4b5aada8cd690716a/users/0670d87cb9284368ad13959c5554b4be/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:28:44.310427 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d76d5a-facb-43a2-bd37-48e954534291 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:44.310720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d76d5a-facb-43a2-bd37-48e954534291 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:44.311057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d76d5a-facb-43a2-bd37-48e954534291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:44.341697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-31d76d5a-facb-43a2-bd37-48e954534291 None None] MFA Rules not processed for user `0670d87cb9284368ad13959c5554b4be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:28:44.342920 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-31d76d5a-facb-43a2-bd37-48e954534291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:28:44.407802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 744/1502] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:28:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:28:44.422020 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d942f34-9e2a-4164-9f20-320fcc17b847 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:44.422302 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d942f34-9e2a-4164-9f20-320fcc17b847 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:44.422583 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d942f34-9e2a-4164-9f20-320fcc17b847 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:44.423971 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 759/1503] 10.222.0.62 () {54 vars in 964 bytes} [Wed Sep 7 13:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:28:44.429287 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7a503-c269-4cf9-8105-19ebce8f66e4 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:44.429573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7a503-c269-4cf9-8105-19ebce8f66e4 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:44.429848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7a503-c269-4cf9-8105-19ebce8f66e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:44.494899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-f6c7a503-c269-4cf9-8105-19ebce8f66e4 None None] MFA Rules not processed for user `fc2fa0fa46b842ceb1cac437166b8aea`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:28:44.496173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f6c7a503-c269-4cf9-8105-19ebce8f66e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:28:44.577281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 745/1504] 10.222.0.62 () {58 vars in 1061 bytes} [Wed Sep 7 13:28:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:28:44.587504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d942f34-9e2a-4164-9f20-320fcc17b847 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:28:44.598413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d942f34-9e2a-4164-9f20-320fcc17b847 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:28:44.603253 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1966a9-e1a0-4d32-b0bb-8d28bc632d8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:28:44.603870 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1966a9-e1a0-4d32-b0bb-8d28bc632d8c service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:44.604149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1966a9-e1a0-4d32-b0bb-8d28bc632d8c service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:44.604420 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1966a9-e1a0-4d32-b0bb-8d28bc632d8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:44.613545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1966a9-e1a0-4d32-b0bb-8d28bc632d8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:28:44.616074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1966a9-e1a0-4d32-b0bb-8d28bc632d8c service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:28:44.624889 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 760/1505] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:28:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:28:44.673729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9af15e-f333-4d92-82ba-d9d9aaa2f36d None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:44.682824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9af15e-f333-4d92-82ba-d9d9aaa2f36d None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:44.682824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9af15e-f333-4d92-82ba-d9d9aaa2f36d None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:44.682824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 746/1506] 10.222.0.62 () {52 vars in 830 bytes} [Wed Sep 7 13:28:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:28:44.686218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1966a9-e1a0-4d32-b0bb-8d28bc632d8c service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:28:44.755283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1966a9-e1a0-4d32-b0bb-8d28bc632d8c service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:28:44.760383 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c579b-458c-4daf-97f2-c1a31a48377c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:28:44.761016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c579b-458c-4daf-97f2-c1a31a48377c admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:44.761292 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c579b-458c-4daf-97f2-c1a31a48377c admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:44.761558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c579b-458c-4daf-97f2-c1a31a48377c admin admin] PATH_INFO: `/v3/users/0670d87cb9284368ad13959c5554b4be` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:44.762707 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c579b-458c-4daf-97f2-c1a31a48377c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0670d87cb9284368ad13959c5554b4be)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:28:44.763740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: WARNING py.warnings [None req-c76c579b-458c-4daf-97f2-c1a31a48377c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:28:44.763740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: warnings.warn(msg) Sep 07 13:28:44.763740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: Sep 07 13:28:44.764922 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c579b-458c-4daf-97f2-c1a31a48377c admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:28:45.025964 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c76c579b-458c-4daf-97f2-c1a31a48377c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0670d87cb9284368ad13959c5554b4be'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:28:45.099636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c76c579b-458c-4daf-97f2-c1a31a48377c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0670d87cb9284368ad13959c5554b4be'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:28:45.101046 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c76c579b-458c-4daf-97f2-c1a31a48377c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0670d87cb9284368ad13959c5554b4be'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:28:45.101671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c76c579b-458c-4daf-97f2-c1a31a48377c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0670d87cb9284368ad13959c5554b4be'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:28:45.118228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c76c579b-458c-4daf-97f2-c1a31a48377c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0670d87cb9284368ad13959c5554b4be'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:28:45.140847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:28:45.150129 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 761/1507] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:28:44 2022] DELETE /identity/v3/users/0670d87cb9284368ad13959c5554b4be => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:28:45.154701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481a23f-6405-47e4-bec7-f77f6bc587ec None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:45.154995 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481a23f-6405-47e4-bec7-f77f6bc587ec None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:45.155308 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481a23f-6405-47e4-bec7-f77f6bc587ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:45.176918 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-9481a23f-6405-47e4-bec7-f77f6bc587ec None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:28:45.179290 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9481a23f-6405-47e4-bec7-f77f6bc587ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:28:45.189340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:28:45.240983 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 747/1508] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:28:45.252052 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c579b-458c-4daf-97f2-c1a31a48377c admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:28:45.253664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c76c579b-458c-4daf-97f2-c1a31a48377c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:28:45.340377 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c579b-458c-4daf-97f2-c1a31a48377c admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:28:45.344166 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaee951-da28-4302-9831-f9c871dec6a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:28:45.344788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaee951-da28-4302-9831-f9c871dec6a7 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:45.345070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaee951-da28-4302-9831-f9c871dec6a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:45.345347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaee951-da28-4302-9831-f9c871dec6a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:45.355651 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaee951-da28-4302-9831-f9c871dec6a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:28:45.357091 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaee951-da28-4302-9831-f9c871dec6a7 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:28:45.367319 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 762/1509] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:28:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:28:45.877896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481a23f-6405-47e4-bec7-f77f6bc587ec None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:28:45.879447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9481a23f-6405-47e4-bec7-f77f6bc587ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:28:45.892187 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481a23f-6405-47e4-bec7-f77f6bc587ec None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:28:45.898318 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ad556-2636-4047-b768-a32ddab04140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:28:45.899043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ad556-2636-4047-b768-a32ddab04140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:28:45.899334 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ad556-2636-4047-b768-a32ddab04140 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:28:45.899631 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ad556-2636-4047-b768-a32ddab04140 admin admin] PATH_INFO: `/v3/projects/e803464281e645d4b5aada8cd690716a` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:28:45.900804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ad556-2636-4047-b768-a32ddab04140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e803464281e645d4b5aada8cd690716a)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:28:45.901905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: WARNING py.warnings [None req-e64ad556-2636-4047-b768-a32ddab04140 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:28:45.901905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: warnings.warn(msg) Sep 07 13:28:45.901905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: Sep 07 13:28:45.903408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ad556-2636-4047-b768-a32ddab04140 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:28:46.044220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e64ad556-2636-4047-b768-a32ddab04140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e803464281e645d4b5aada8cd690716a', 'request_id': 'req-e64ad556-2636-4047-b768-a32ddab04140'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:28:46.059453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e64ad556-2636-4047-b768-a32ddab04140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e803464281e645d4b5aada8cd690716a', 'request_id': 'req-e64ad556-2636-4047-b768-a32ddab04140'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:28:46.139687 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e64ad556-2636-4047-b768-a32ddab04140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e803464281e645d4b5aada8cd690716a', 'request_id': 'req-e64ad556-2636-4047-b768-a32ddab04140'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:28:46.153721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e64ad556-2636-4047-b768-a32ddab04140 admin admin] The token cache is being invalidate because project e803464281e645d4b5aada8cd690716a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:28:46.154041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e64ad556-2636-4047-b768-a32ddab04140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:28:46.154832 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e64ad556-2636-4047-b768-a32ddab04140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:28:46.156491 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 748/1510] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:28:45 2022] DELETE /identity/v3/projects/e803464281e645d4b5aada8cd690716a => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:28:55.375715 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:28:56.150230 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:30:20.578036 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09dd08f-abf0-4b5c-98fa-59c8b9fc8f73 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:20.578647 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09dd08f-abf0-4b5c-98fa-59c8b9fc8f73 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:20.578750 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09dd08f-abf0-4b5c-98fa-59c8b9fc8f73 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:20.580554 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:30:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:30:20.586509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67705ef6-052d-4ca4-bb51-d3038c5f9f0b None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:20.587643 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67705ef6-052d-4ca4-bb51-d3038c5f9f0b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:20.588098 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67705ef6-052d-4ca4-bb51-d3038c5f9f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:20.609367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-67705ef6-052d-4ca4-bb51-d3038c5f9f0b None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:30:20.610580 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-67705ef6-052d-4ca4-bb51-d3038c5f9f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:20.674932 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:30:20.834120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09dd08f-abf0-4b5c-98fa-59c8b9fc8f73 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:30:20.836093 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b09dd08f-abf0-4b5c-98fa-59c8b9fc8f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:20.945804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09dd08f-abf0-4b5c-98fa-59c8b9fc8f73 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:30:20.952325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476798fd-7ac7-4599-8d7a-4e529f79ec82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:30:20.952936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476798fd-7ac7-4599-8d7a-4e529f79ec82 service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:20.953216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476798fd-7ac7-4599-8d7a-4e529f79ec82 service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:20.953485 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476798fd-7ac7-4599-8d7a-4e529f79ec82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:20.963034 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476798fd-7ac7-4599-8d7a-4e529f79ec82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:30:20.964787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476798fd-7ac7-4599-8d7a-4e529f79ec82 service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:30:20.976500 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 764/1513] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:30:21.922431 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c7125-2a5f-4bfb-9931-b7a9b57bdc3f None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:21.923027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c7125-2a5f-4bfb-9931-b7a9b57bdc3f None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:21.923027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c7125-2a5f-4bfb-9931-b7a9b57bdc3f None None] PATH_INFO: `/v3` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:21.924720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 750/1514] 10.222.0.74 () {54 vars in 925 bytes} [Wed Sep 7 13:30:21 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:30:21.931800 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20fddf-e16c-4378-b51f-4c89ccf09273 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:21.932340 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20fddf-e16c-4378-b51f-4c89ccf09273 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:21.932340 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20fddf-e16c-4378-b51f-4c89ccf09273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:21.994935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-2d20fddf-e16c-4378-b51f-4c89ccf09273 None None] MFA Rules not processed for user `bee90bf951f04c2e8e19ceeacb8cfcb2`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:30:21.996238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-2d20fddf-e16c-4378-b51f-4c89ccf09273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:22.082956 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 765/1515] 10.222.0.74 () {58 vars in 1015 bytes} [Wed Sep 7 13:30:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:30:22.122778 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a33578-5658-4034-8b2c-c1b3f78ab689 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:22.123145 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a33578-5658-4034-8b2c-c1b3f78ab689 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:22.123380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a33578-5658-4034-8b2c-c1b3f78ab689 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:22.124717 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 751/1516] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:30:22.131493 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b52838-8455-46cf-9758-adc2d7bc7755 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:22.131801 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b52838-8455-46cf-9758-adc2d7bc7755 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:22.132075 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b52838-8455-46cf-9758-adc2d7bc7755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:22.152368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-92b52838-8455-46cf-9758-adc2d7bc7755 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:30:22.153575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-92b52838-8455-46cf-9758-adc2d7bc7755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:22.161706 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 766/1517] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:30:22.320572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a33578-5658-4034-8b2c-c1b3f78ab689 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:30:22.342547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a33578-5658-4034-8b2c-c1b3f78ab689 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:30:22.348243 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9134822-848d-4914-99e9-e480e03cfbf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:30:22.348869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9134822-848d-4914-99e9-e480e03cfbf9 service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:22.349146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9134822-848d-4914-99e9-e480e03cfbf9 service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:22.349415 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9134822-848d-4914-99e9-e480e03cfbf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:22.359489 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9134822-848d-4914-99e9-e480e03cfbf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:30:22.361091 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9134822-848d-4914-99e9-e480e03cfbf9 service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:30:22.372173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 752/1518] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:30:30.882370 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dcdf8-debb-47d5-b063-29dc845f8fb7 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:30.883106 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dcdf8-debb-47d5-b063-29dc845f8fb7 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:30.883106 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dcdf8-debb-47d5-b063-29dc845f8fb7 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:30.884323 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 767/1519] 10.222.0.62 () {54 vars in 961 bytes} [Wed Sep 7 13:30:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:30:30.890087 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d6898-7e5f-490f-8050-3ef05571e13c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:30.890599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d6898-7e5f-490f-8050-3ef05571e13c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:30.890703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d6898-7e5f-490f-8050-3ef05571e13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:30.912672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-718d6898-7e5f-490f-8050-3ef05571e13c None None] MFA Rules not processed for user `bee90bf951f04c2e8e19ceeacb8cfcb2`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:30:30.913908 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-718d6898-7e5f-490f-8050-3ef05571e13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:30.924549 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 753/1520] 10.222.0.62 () {58 vars in 1058 bytes} [Wed Sep 7 13:30:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:30:30.932569 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dcdf8-debb-47d5-b063-29dc845f8fb7 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:30:30.947858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dcdf8-debb-47d5-b063-29dc845f8fb7 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:30:30.953496 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c304c-095b-44d8-a51b-ce14359652af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bee90bf951f04c2e8e19ceeacb8cfcb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:30:30.954197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c304c-095b-44d8-a51b-ce14359652af service placement] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:30.954554 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c304c-095b-44d8-a51b-ce14359652af service placement] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:30.954867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c304c-095b-44d8-a51b-ce14359652af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:30.966101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c304c-095b-44d8-a51b-ce14359652af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:30:30.968229 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c304c-095b-44d8-a51b-ce14359652af service placement] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:30:30.983157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 768/1521] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:30:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:30:33.907017 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ffd1a-7b73-4dca-954c-9192f6bd5d34 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:33.907848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ffd1a-7b73-4dca-954c-9192f6bd5d34 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:33.908041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ffd1a-7b73-4dca-954c-9192f6bd5d34 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:33.909516 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:30:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:30:33.918177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1298d99-304a-4d89-8238-fc72d1cac950 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:33.918873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1298d99-304a-4d89-8238-fc72d1cac950 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:33.918873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1298d99-304a-4d89-8238-fc72d1cac950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:33.927510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:30:33.942802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b1298d99-304a-4d89-8238-fc72d1cac950 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:30:33.944273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b1298d99-304a-4d89-8238-fc72d1cac950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:33.954290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:30:34.124598 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ffd1a-7b73-4dca-954c-9192f6bd5d34 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:30:34.141045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ffd1a-7b73-4dca-954c-9192f6bd5d34 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:30:34.147376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a709355-41a1-42e0-ba10-a552fae29a57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:30:34.147999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a709355-41a1-42e0-ba10-a552fae29a57 service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:34.148276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a709355-41a1-42e0-ba10-a552fae29a57 service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:34.148546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a709355-41a1-42e0-ba10-a552fae29a57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:34.159757 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a709355-41a1-42e0-ba10-a552fae29a57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:30:34.161424 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a709355-41a1-42e0-ba10-a552fae29a57 service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:30:34.173680 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 755/1524] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:30:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:30:35.329477 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1388e4-00f3-4fc5-bea6-d559bdd216a9 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:35.330339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1388e4-00f3-4fc5-bea6-d559bdd216a9 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:35.330339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1388e4-00f3-4fc5-bea6-d559bdd216a9 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:35.331956 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 13:30:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:30:35.343159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba776fc3-7ea5-4403-939f-7af3d2f9a327 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:35.343725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba776fc3-7ea5-4403-939f-7af3d2f9a327 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:35.343725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba776fc3-7ea5-4403-939f-7af3d2f9a327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:35.368137 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-ba776fc3-7ea5-4403-939f-7af3d2f9a327 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:30:35.369438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ba776fc3-7ea5-4403-939f-7af3d2f9a327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:35.377437 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 13:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:30:35.533445 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1388e4-00f3-4fc5-bea6-d559bdd216a9 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:30:35.554171 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1388e4-00f3-4fc5-bea6-d559bdd216a9 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:30:35.560215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ecaf0c-61d6-44a2-9dc1-e5731096a36f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:30:35.560855 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecaf0c-61d6-44a2-9dc1-e5731096a36f service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:35.561135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecaf0c-61d6-44a2-9dc1-e5731096a36f service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:35.561410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecaf0c-61d6-44a2-9dc1-e5731096a36f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:35.578672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ecaf0c-61d6-44a2-9dc1-e5731096a36f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:30:35.580649 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ecaf0c-61d6-44a2-9dc1-e5731096a36f service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:30:35.592510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 771/1527] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:30:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:30:39.767561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be25946-4550-4de3-be71-6d874b98e815 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:39.768112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be25946-4550-4de3-be71-6d874b98e815 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:39.768112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be25946-4550-4de3-be71-6d874b98e815 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:39.769416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:30:39.774644 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc65a4-8c07-4903-ae04-b282fe225da9 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:39.775317 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc65a4-8c07-4903-ae04-b282fe225da9 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:39.775317 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc65a4-8c07-4903-ae04-b282fe225da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:39.795134 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-defc65a4-8c07-4903-ae04-b282fe225da9 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:30:39.796337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-defc65a4-8c07-4903-ae04-b282fe225da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:39.804443 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:30:40.100692 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be25946-4550-4de3-be71-6d874b98e815 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:30:40.102463 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6be25946-4550-4de3-be71-6d874b98e815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:40.171744 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be25946-4550-4de3-be71-6d874b98e815 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:30:40.175751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ddd11-062c-4587-9669-75a0be7cbdf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:30:40.176380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ddd11-062c-4587-9669-75a0be7cbdf6 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:40.176750 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ddd11-062c-4587-9669-75a0be7cbdf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:40.177029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ddd11-062c-4587-9669-75a0be7cbdf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:40.187424 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ddd11-062c-4587-9669-75a0be7cbdf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:30:40.188931 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ddd11-062c-4587-9669-75a0be7cbdf6 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:30:40.202165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 758/1530] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:30:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:30:42.483329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c8a0e-2304-4989-8bcb-91988cda1657 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:42.483904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c8a0e-2304-4989-8bcb-91988cda1657 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:42.483904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c8a0e-2304-4989-8bcb-91988cda1657 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:42.485230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:30:42.491676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a1c51-0847-498e-a28d-568172609982 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:42.492279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a1c51-0847-498e-a28d-568172609982 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:42.492590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a1c51-0847-498e-a28d-568172609982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:42.520244 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-1b5a1c51-0847-498e-a28d-568172609982 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:30:42.521554 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-1b5a1c51-0847-498e-a28d-568172609982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:42.534126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:30:42.843186 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c8a0e-2304-4989-8bcb-91988cda1657 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:30:42.845205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c26c8a0e-2304-4989-8bcb-91988cda1657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:42.862590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c8a0e-2304-4989-8bcb-91988cda1657 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:30:42.866479 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6089561-f3a7-4446-a57b-70a126fe9c26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:30:42.867147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6089561-f3a7-4446-a57b-70a126fe9c26 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:42.867463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6089561-f3a7-4446-a57b-70a126fe9c26 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:42.867733 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6089561-f3a7-4446-a57b-70a126fe9c26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:42.883177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6089561-f3a7-4446-a57b-70a126fe9c26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:30:42.884870 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6089561-f3a7-4446-a57b-70a126fe9c26 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:30:42.897841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 774/1533] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:30:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:30:46.830189 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c04864-4180-4c06-9303-a91f0ec04df6 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:46.830743 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c04864-4180-4c06-9303-a91f0ec04df6 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:46.830854 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c04864-4180-4c06-9303-a91f0ec04df6 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:46.832154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:30:46.838789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd7845-c52a-464c-8e07-85fa08df3632 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:46.839347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd7845-c52a-464c-8e07-85fa08df3632 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:46.839458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd7845-c52a-464c-8e07-85fa08df3632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:46.863157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-7bbd7845-c52a-464c-8e07-85fa08df3632 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:30:46.864465 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7bbd7845-c52a-464c-8e07-85fa08df3632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:46.874020 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:30:47.193295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c04864-4180-4c06-9303-a91f0ec04df6 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:30:47.195234 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-99c04864-4180-4c06-9303-a91f0ec04df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:47.202042 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=75 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:30:47.263633 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c04864-4180-4c06-9303-a91f0ec04df6 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:30:47.267335 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd635c5c-014d-4e10-a09c-0b331e45ef69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:30:47.267980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd635c5c-014d-4e10-a09c-0b331e45ef69 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:47.268311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd635c5c-014d-4e10-a09c-0b331e45ef69 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:47.268900 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd635c5c-014d-4e10-a09c-0b331e45ef69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:47.279984 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd635c5c-014d-4e10-a09c-0b331e45ef69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:30:47.281582 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd635c5c-014d-4e10-a09c-0b331e45ef69 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:30:47.297072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 761/1536] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:30:49.561395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff30f-7427-4e16-968f-a5e1f3687e33 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:49.565754 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff30f-7427-4e16-968f-a5e1f3687e33 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:49.565754 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff30f-7427-4e16-968f-a5e1f3687e33 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:49.565754 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 13:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:30:49.571571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc21c9-ce64-48b9-9818-acaedf2c2406 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:49.571571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc21c9-ce64-48b9-9818-acaedf2c2406 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:49.571571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc21c9-ce64-48b9-9818-acaedf2c2406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:49.595320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c0cc21c9-ce64-48b9-9818-acaedf2c2406 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:30:49.596560 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c0cc21c9-ce64-48b9-9818-acaedf2c2406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:49.606720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 13:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:30:49.937386 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eff30f-7427-4e16-968f-a5e1f3687e33 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:30:49.939340 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-92eff30f-7427-4e16-968f-a5e1f3687e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:30:49.955848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eff30f-7427-4e16-968f-a5e1f3687e33 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:30:49.961789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d5d21-d5d5-4da6-9b15-f16a8c738378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:30:49.962793 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d5d21-d5d5-4da6-9b15-f16a8c738378 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:30:49.963322 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d5d21-d5d5-4da6-9b15-f16a8c738378 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:30:49.963813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d5d21-d5d5-4da6-9b15-f16a8c738378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:30:49.981282 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d5d21-d5d5-4da6-9b15-f16a8c738378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:30:49.984016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d5d21-d5d5-4da6-9b15-f16a8c738378 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:30:50.004673 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 777/1539] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:30:59.588790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=28 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:30:59.995525 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=26 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:19.657202 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845299b-c5bf-4272-859b-c70d2d154a3d None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:19.659263 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845299b-c5bf-4272-859b-c70d2d154a3d None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:19.659263 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845299b-c5bf-4272-859b-c70d2d154a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:19.694332 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-1845299b-c5bf-4272-859b-c70d2d154a3d None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:19.696038 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-1845299b-c5bf-4272-859b-c70d2d154a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:19.711180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 763/1540] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:32:19.739688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ca36b-6d3b-487c-87bc-844294985e19 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:19.740620 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ca36b-6d3b-487c-87bc-844294985e19 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:19.740620 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ca36b-6d3b-487c-87bc-844294985e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:19.750502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bad96c-1c3b-4d20-b69e-8e378ea68d8a None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:19.750821 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bad96c-1c3b-4d20-b69e-8e378ea68d8a None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:19.751212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bad96c-1c3b-4d20-b69e-8e378ea68d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:19.809121 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b91ca36b-6d3b-487c-87bc-844294985e19 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:19.812572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b91ca36b-6d3b-487c-87bc-844294985e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:19.836922 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c6bad96c-1c3b-4d20-b69e-8e378ea68d8a None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:19.839288 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c6bad96c-1c3b-4d20-b69e-8e378ea68d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:19.850135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 778/1541] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:19.853171 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba7a51-5a37-4d5d-abb9-35f28df7f479 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:19.853638 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba7a51-5a37-4d5d-abb9-35f28df7f479 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:19.854078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba7a51-5a37-4d5d-abb9-35f28df7f479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:19.872821 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 764/1542] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:19.875760 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bad96c-1c3b-4d20-b69e-8e378ea68d8a None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:19.910374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bad96c-1c3b-4d20-b69e-8e378ea68d8a None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:19.915617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-daba7a51-5a37-4d5d-abb9-35f28df7f479 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:19.920231 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-daba7a51-5a37-4d5d-abb9-35f28df7f479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:19.928165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa22bd-0c21-44bc-ad3f-67be99483db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:19.929345 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa22bd-0c21-44bc-ad3f-67be99483db0 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:19.929846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa22bd-0c21-44bc-ad3f-67be99483db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:19.930296 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa22bd-0c21-44bc-ad3f-67be99483db0 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:19.931268 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa22bd-0c21-44bc-ad3f-67be99483db0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:32:19.931595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa22bd-0c21-44bc-ad3f-67be99483db0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:19.933616 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa22bd-0c21-44bc-ad3f-67be99483db0 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:19.967650 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 779/1543] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:19.970383 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db5572-ebdc-47ed-9a6a-a5ccb8036125 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:19.970889 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db5572-ebdc-47ed-9a6a-a5ccb8036125 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:19.971399 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db5572-ebdc-47ed-9a6a-a5ccb8036125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:19.982897 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 765/1544] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:32:19.990647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73762110-3c34-4715-9915-edf8b9dad528 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:19.990895 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73762110-3c34-4715-9915-edf8b9dad528 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:19.991262 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73762110-3c34-4715-9915-edf8b9dad528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.025041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-73762110-3c34-4715-9915-edf8b9dad528 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:20.027574 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-73762110-3c34-4715-9915-edf8b9dad528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:20.035978 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-23db5572-ebdc-47ed-9a6a-a5ccb8036125 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:20.036237 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-23db5572-ebdc-47ed-9a6a-a5ccb8036125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:20.046179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 766/1545] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:20.049844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06287845-64a2-4cc7-bbab-f0d19ab168ec None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.050198 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06287845-64a2-4cc7-bbab-f0d19ab168ec None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.050485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06287845-64a2-4cc7-bbab-f0d19ab168ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.060744 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 780/1546] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:20.062551 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db5572-ebdc-47ed-9a6a-a5ccb8036125 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:20.089452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db5572-ebdc-47ed-9a6a-a5ccb8036125 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:20.100939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bc54f-b5e3-42c8-87b3-cf8b93b543ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:20.101691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-06287845-64a2-4cc7-bbab-f0d19ab168ec None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:20.101992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bc54f-b5e3-42c8-87b3-cf8b93b543ab admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.102381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bc54f-b5e3-42c8-87b3-cf8b93b543ab admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.102879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bc54f-b5e3-42c8-87b3-cf8b93b543ab admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.103100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-06287845-64a2-4cc7-bbab-f0d19ab168ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:20.104498 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bc54f-b5e3-42c8-87b3-cf8b93b543ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:20.109188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bc54f-b5e3-42c8-87b3-cf8b93b543ab admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:20.114089 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 767/1547] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:20.115367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a629b6-cecc-4baf-8ac3-172509b4f5d2 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.115691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a629b6-cecc-4baf-8ac3-172509b4f5d2 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.116072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a629b6-cecc-4baf-8ac3-172509b4f5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.157918 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-f0a629b6-cecc-4baf-8ac3-172509b4f5d2 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:20.160332 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f0a629b6-cecc-4baf-8ac3-172509b4f5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:20.176008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 768/1548] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:20.177274 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a629b6-cecc-4baf-8ac3-172509b4f5d2 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:20.192846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a629b6-cecc-4baf-8ac3-172509b4f5d2 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:20.199963 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800445c-52da-4d9d-8f6e-029a1df85bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:20.201602 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800445c-52da-4d9d-8f6e-029a1df85bec admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.201996 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800445c-52da-4d9d-8f6e-029a1df85bec admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.202280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800445c-52da-4d9d-8f6e-029a1df85bec admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.203017 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800445c-52da-4d9d-8f6e-029a1df85bec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:32:20.203571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800445c-52da-4d9d-8f6e-029a1df85bec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:20.207640 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800445c-52da-4d9d-8f6e-029a1df85bec admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:20.228363 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 769/1549] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:32:20.230573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52ce33-ca27-491c-a6d4-17573ca5918d None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.231096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52ce33-ca27-491c-a6d4-17573ca5918d None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.231641 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52ce33-ca27-491c-a6d4-17573ca5918d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.256901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-8c52ce33-ca27-491c-a6d4-17573ca5918d None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:20.258124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-8c52ce33-ca27-491c-a6d4-17573ca5918d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:20.272638 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 781/1550] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/projects => generated 393 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:20.274196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b378ae8-99c3-4a62-a962-af5bd9d26a17 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.274646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b378ae8-99c3-4a62-a962-af5bd9d26a17 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.274812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b378ae8-99c3-4a62-a962-af5bd9d26a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.277235 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 770/1551] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:20.279642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870006d-be1c-4a8f-a462-a9ae6a363c68 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.279642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870006d-be1c-4a8f-a462-a9ae6a363c68 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.279891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870006d-be1c-4a8f-a462-a9ae6a363c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.303852 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-3b378ae8-99c3-4a62-a962-af5bd9d26a17 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:20.305756 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-f870006d-be1c-4a8f-a462-a9ae6a363c68 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:20.305905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-3b378ae8-99c3-4a62-a962-af5bd9d26a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:20.308172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f870006d-be1c-4a8f-a462-a9ae6a363c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:20.398184 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 771/1552] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:20.400094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdede29-4248-415c-8a38-384beef42d82 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.400592 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdede29-4248-415c-8a38-384beef42d82 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.401069 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdede29-4248-415c-8a38-384beef42d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.413989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 782/1553] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:20.415585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b378ae8-99c3-4a62-a962-af5bd9d26a17 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:20.430305 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-7cdede29-4248-415c-8a38-384beef42d82 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:20.432284 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-7cdede29-4248-415c-8a38-384beef42d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:20.437579 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b378ae8-99c3-4a62-a962-af5bd9d26a17 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:20.442740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ec841-b2f7-4c87-8eda-54223ec0b9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:20.443436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ec841-b2f7-4c87-8eda-54223ec0b9da admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.443715 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ec841-b2f7-4c87-8eda-54223ec0b9da admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.443986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ec841-b2f7-4c87-8eda-54223ec0b9da admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.444667 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ec841-b2f7-4c87-8eda-54223ec0b9da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:32:20.444969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ec841-b2f7-4c87-8eda-54223ec0b9da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:20.446025 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: WARNING py.warnings [None req-ac4ec841-b2f7-4c87-8eda-54223ec0b9da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:32:20.446025 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: warnings.warn(msg) Sep 07 13:32:20.446025 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: Sep 07 13:32:20.446543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 772/1554] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:20.446940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ec841-b2f7-4c87-8eda-54223ec0b9da admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:20.448501 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef29d4-3e85-44b4-8d75-0ad5f3402129 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.449004 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef29d4-3e85-44b4-8d75-0ad5f3402129 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.449427 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef29d4-3e85-44b4-8d75-0ad5f3402129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.467515 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 783/1555] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:32:20.469416 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ec841-b2f7-4c87-8eda-54223ec0b9da admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:20.485562 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-00ef29d4-3e85-44b4-8d75-0ad5f3402129 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:20.487495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-00ef29d4-3e85-44b4-8d75-0ad5f3402129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:20.489581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ec841-b2f7-4c87-8eda-54223ec0b9da admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:20.497909 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77033b0b-cbd0-4533-bb5c-aae92873fec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:20.498522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77033b0b-cbd0-4533-bb5c-aae92873fec4 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.498808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77033b0b-cbd0-4533-bb5c-aae92873fec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.499308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77033b0b-cbd0-4533-bb5c-aae92873fec4 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.499765 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77033b0b-cbd0-4533-bb5c-aae92873fec4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:20.500833 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 773/1556] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:20.501334 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef29d4-3e85-44b4-8d75-0ad5f3402129 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:20.502087 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77033b0b-cbd0-4533-bb5c-aae92873fec4 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:20.519116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef29d4-3e85-44b4-8d75-0ad5f3402129 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:20.526579 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda81c81-4910-43d4-b869-34739a93b3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:20.527262 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81c81-4910-43d4-b869-34739a93b3cf admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.527548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81c81-4910-43d4-b869-34739a93b3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.528987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81c81-4910-43d4-b869-34739a93b3cf admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.529452 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda81c81-4910-43d4-b869-34739a93b3cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:20.531790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda81c81-4910-43d4-b869-34739a93b3cf admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:20.671617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 784/1557] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/projects => generated 375 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:20.671617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77033b0b-cbd0-4533-bb5c-aae92873fec4 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:20.708280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 774/1558] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:20.709588 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda81c81-4910-43d4-b869-34739a93b3cf admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:20.723829 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda81c81-4910-43d4-b869-34739a93b3cf admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:20.767434 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77033b0b-cbd0-4533-bb5c-aae92873fec4 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:20.775344 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fee66-cc93-4b3f-b427-2424258c7af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:20.776001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fee66-cc93-4b3f-b427-2424258c7af5 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.776286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fee66-cc93-4b3f-b427-2424258c7af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.776553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fee66-cc93-4b3f-b427-2424258c7af5 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.777218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650fee66-cc93-4b3f-b427-2424258c7af5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:32:20.777513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650fee66-cc93-4b3f-b427-2424258c7af5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:20.780119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650fee66-cc93-4b3f-b427-2424258c7af5 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:20.794679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 785/1559] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:32:20.797080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4170f7-601b-4ea6-8d7d-094024377398 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.797392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4170f7-601b-4ea6-8d7d-094024377398 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.797737 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4170f7-601b-4ea6-8d7d-094024377398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.812765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ca80b-65a5-425f-9b92-85bda273d954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:20.813835 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ca80b-65a5-425f-9b92-85bda273d954 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.814353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ca80b-65a5-425f-9b92-85bda273d954 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.814828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ca80b-65a5-425f-9b92-85bda273d954 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.816035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ca80b-65a5-425f-9b92-85bda273d954 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:32:20.816507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ca80b-65a5-425f-9b92-85bda273d954 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:20.819795 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ca80b-65a5-425f-9b92-85bda273d954 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:20.825069 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-0f4170f7-601b-4ea6-8d7d-094024377398 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:20.826919 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0f4170f7-601b-4ea6-8d7d-094024377398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:20.829924 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:20.837412 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 775/1560] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:32:20.847546 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 786/1561] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:20.847546 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4170f7-601b-4ea6-8d7d-094024377398 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:20.847810 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38047f83-3f0f-48db-a29f-c031abf76cde None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.847810 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38047f83-3f0f-48db-a29f-c031abf76cde None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.847810 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38047f83-3f0f-48db-a29f-c031abf76cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.852945 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4170f7-601b-4ea6-8d7d-094024377398 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:20.858873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c9746-0b1b-465c-b077-f108e4455df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:20.859744 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c9746-0b1b-465c-b077-f108e4455df2 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.860076 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c9746-0b1b-465c-b077-f108e4455df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.860407 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c9746-0b1b-465c-b077-f108e4455df2 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.861232 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c9746-0b1b-465c-b077-f108e4455df2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:20.863883 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c9746-0b1b-465c-b077-f108e4455df2 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:20.876547 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:20.880228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-38047f83-3f0f-48db-a29f-c031abf76cde None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:20.881746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-38047f83-3f0f-48db-a29f-c031abf76cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:20.896793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 776/1562] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:20.900020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccc3ab-05fa-41fa-a4d6-03b627c0d6e0 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.900020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccc3ab-05fa-41fa-a4d6-03b627c0d6e0 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.900020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccc3ab-05fa-41fa-a4d6-03b627c0d6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.923295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-21ccc3ab-05fa-41fa-a4d6-03b627c0d6e0 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:20.924531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-21ccc3ab-05fa-41fa-a4d6-03b627c0d6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:20.934021 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 777/1563] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:20.935677 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccc3ab-05fa-41fa-a4d6-03b627c0d6e0 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:20.947016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccc3ab-05fa-41fa-a4d6-03b627c0d6e0 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:20.980004 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586120e8-5c32-48ce-bc32-92b8f35fa465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:20.980004 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586120e8-5c32-48ce-bc32-92b8f35fa465 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:20.980004 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586120e8-5c32-48ce-bc32-92b8f35fa465 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:20.980004 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586120e8-5c32-48ce-bc32-92b8f35fa465 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:20.980541 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586120e8-5c32-48ce-bc32-92b8f35fa465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:20.980541 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586120e8-5c32-48ce-bc32-92b8f35fa465 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:21.079543 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 787/1564] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/projects => generated 373 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:21.080349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c9746-0b1b-465c-b077-f108e4455df2 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:21.105247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 778/1565] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:32:20 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:21.106781 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586120e8-5c32-48ce-bc32-92b8f35fa465 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:21.117761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586120e8-5c32-48ce-bc32-92b8f35fa465 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:21.162847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c9746-0b1b-465c-b077-f108e4455df2 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:21.168342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1083aaf-cd11-441d-a732-a401123fc509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:21.169018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1083aaf-cd11-441d-a732-a401123fc509 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:21.169475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1083aaf-cd11-441d-a732-a401123fc509 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:21.169739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1083aaf-cd11-441d-a732-a401123fc509 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:21.170662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1083aaf-cd11-441d-a732-a401123fc509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:21.172823 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1083aaf-cd11-441d-a732-a401123fc509 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:21.185827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 788/1566] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:21.185827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1083aaf-cd11-441d-a732-a401123fc509 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:21.196921 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1083aaf-cd11-441d-a732-a401123fc509 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:21.205161 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b86e6e-64c4-4443-b3f2-73602048c032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:21.205894 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b86e6e-64c4-4443-b3f2-73602048c032 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:21.206372 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b86e6e-64c4-4443-b3f2-73602048c032 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:21.206998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b86e6e-64c4-4443-b3f2-73602048c032 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:21.207850 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b86e6e-64c4-4443-b3f2-73602048c032 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:21.210252 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b86e6e-64c4-4443-b3f2-73602048c032 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:21.216659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a020c85-257a-40b1-a342-ddd3eb33fb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:21.220047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a020c85-257a-40b1-a342-ddd3eb33fb82 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:21.220047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a020c85-257a-40b1-a342-ddd3eb33fb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:21.220047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a020c85-257a-40b1-a342-ddd3eb33fb82 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:21.220047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a020c85-257a-40b1-a342-ddd3eb33fb82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:21.220047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: WARNING py.warnings [None req-8a020c85-257a-40b1-a342-ddd3eb33fb82 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:32:21.220047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: warnings.warn(msg) Sep 07 13:32:21.220047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: Sep 07 13:32:21.221099 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a020c85-257a-40b1-a342-ddd3eb33fb82 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:21.369706 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 789/1567] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:32:21 2022] POST /identity/v3/projects => generated 373 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:21.369706 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b86e6e-64c4-4443-b3f2-73602048c032 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:21.437194 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 779/1568] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:32:21 2022] POST /identity/v3/projects => generated 365 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:21.438998 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a020c85-257a-40b1-a342-ddd3eb33fb82 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:21.447668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b86e6e-64c4-4443-b3f2-73602048c032 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:21.547814 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca630022-fd12-49db-b355-bf0ee770c1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:21.548652 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca630022-fd12-49db-b355-bf0ee770c1f7 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:21.548932 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca630022-fd12-49db-b355-bf0ee770c1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:21.549207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca630022-fd12-49db-b355-bf0ee770c1f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:21.550097 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca630022-fd12-49db-b355-bf0ee770c1f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:32:21.550313 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca630022-fd12-49db-b355-bf0ee770c1f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:21.552058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca630022-fd12-49db-b355-bf0ee770c1f7 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:21.558064 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a020c85-257a-40b1-a342-ddd3eb33fb82 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:21.567122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d565b4-b422-4f6d-9085-693c8329d222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:21.567916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d565b4-b422-4f6d-9085-693c8329d222 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:21.570531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d565b4-b422-4f6d-9085-693c8329d222 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:21.570531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d565b4-b422-4f6d-9085-693c8329d222 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:21.570531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d565b4-b422-4f6d-9085-693c8329d222 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:32:21.570531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d565b4-b422-4f6d-9085-693c8329d222 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:21.572225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d565b4-b422-4f6d-9085-693c8329d222 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:21.576176 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 790/1569] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:32:21.577959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca630022-fd12-49db-b355-bf0ee770c1f7 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:21.589886 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:21.592909 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 780/1570] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:32:21.595470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d565b4-b422-4f6d-9085-693c8329d222 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:21.600844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca630022-fd12-49db-b355-bf0ee770c1f7 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:21.612458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bde195-ae57-41d9-b651-0349b5b218fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:21.613707 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bde195-ae57-41d9-b651-0349b5b218fa admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:21.614212 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bde195-ae57-41d9-b651-0349b5b218fa admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:21.614410 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d565b4-b422-4f6d-9085-693c8329d222 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:21.614686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bde195-ae57-41d9-b651-0349b5b218fa admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:21.616157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bde195-ae57-41d9-b651-0349b5b218fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:32:21.616822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bde195-ae57-41d9-b651-0349b5b218fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:21.619602 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d14fa-b3cd-4822-929d-fd45faac5eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:21.620037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bde195-ae57-41d9-b651-0349b5b218fa admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:21.620240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d14fa-b3cd-4822-929d-fd45faac5eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:21.620498 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d14fa-b3cd-4822-929d-fd45faac5eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:21.620767 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d14fa-b3cd-4822-929d-fd45faac5eb3 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:21.621432 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d14fa-b3cd-4822-929d-fd45faac5eb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:21.624205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d14fa-b3cd-4822-929d-fd45faac5eb3 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:21.636967 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 791/1571] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:32:21.638110 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bde195-ae57-41d9-b651-0349b5b218fa admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:21.650575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bde195-ae57-41d9-b651-0349b5b218fa admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:21.660062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab22479-3926-4830-8a8e-83450bcedd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:21.664164 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab22479-3926-4830-8a8e-83450bcedd23 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:21.664164 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab22479-3926-4830-8a8e-83450bcedd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:21.664164 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab22479-3926-4830-8a8e-83450bcedd23 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:21.664164 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab22479-3926-4830-8a8e-83450bcedd23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:21.664164 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab22479-3926-4830-8a8e-83450bcedd23 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:21.676802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 792/1572] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:21.677781 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab22479-3926-4830-8a8e-83450bcedd23 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:21.692611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab22479-3926-4830-8a8e-83450bcedd23 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:21.698796 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b352409-8d2d-4fd0-b99a-9d79adf88a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:21.704276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b352409-8d2d-4fd0-b99a-9d79adf88a17 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:21.704276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b352409-8d2d-4fd0-b99a-9d79adf88a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:21.704276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b352409-8d2d-4fd0-b99a-9d79adf88a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:21.704276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b352409-8d2d-4fd0-b99a-9d79adf88a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:21.704276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b352409-8d2d-4fd0-b99a-9d79adf88a17 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:21.724031 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 793/1573] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:21.725626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b352409-8d2d-4fd0-b99a-9d79adf88a17 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:21.739671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b352409-8d2d-4fd0-b99a-9d79adf88a17 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:21.743016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 781/1574] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:32:21 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:21.743538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d14fa-b3cd-4822-929d-fd45faac5eb3 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:21.748286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e221d-083b-4271-9169-476a003c3d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:21.748912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e221d-083b-4271-9169-476a003c3d62 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:21.749191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e221d-083b-4271-9169-476a003c3d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:21.754057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e221d-083b-4271-9169-476a003c3d62 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:21.754057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e221d-083b-4271-9169-476a003c3d62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:21.754057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: WARNING py.warnings [None req-418e221d-083b-4271-9169-476a003c3d62 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:32:21.754057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: warnings.warn(msg) Sep 07 13:32:21.754057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: Sep 07 13:32:21.754057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e221d-083b-4271-9169-476a003c3d62 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:21.755172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d14fa-b3cd-4822-929d-fd45faac5eb3 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:21.760281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bd66b-a9c2-4e7c-8bfe-7b221c697cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:21.760902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bd66b-a9c2-4e7c-8bfe-7b221c697cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:21.761208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bd66b-a9c2-4e7c-8bfe-7b221c697cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:21.761493 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bd66b-a9c2-4e7c-8bfe-7b221c697cc4 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:21.762145 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bd66b-a9c2-4e7c-8bfe-7b221c697cc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:21.764754 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bd66b-a9c2-4e7c-8bfe-7b221c697cc4 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:21.857703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 794/1575] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:32:21 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:21.859294 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e221d-083b-4271-9169-476a003c3d62 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:21.878727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e221d-083b-4271-9169-476a003c3d62 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:21.881057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 782/1576] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:32:21 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:21.882193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bd66b-a9c2-4e7c-8bfe-7b221c697cc4 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:21.889014 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a67564-6635-416a-bab0-b9ecca7f1eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:21.890082 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a67564-6635-416a-bab0-b9ecca7f1eb5 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:21.890659 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a67564-6635-416a-bab0-b9ecca7f1eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:21.891221 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a67564-6635-416a-bab0-b9ecca7f1eb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:21.892468 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a67564-6635-416a-bab0-b9ecca7f1eb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:21.895619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a67564-6635-416a-bab0-b9ecca7f1eb5 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:21.900000 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bd66b-a9c2-4e7c-8bfe-7b221c697cc4 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:21.906891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379bb4ee-96ff-4c17-ae07-9b75e9bec203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:21.929113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bb4ee-96ff-4c17-ae07-9b75e9bec203 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:21.929113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bb4ee-96ff-4c17-ae07-9b75e9bec203 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:21.929113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bb4ee-96ff-4c17-ae07-9b75e9bec203 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:21.929113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379bb4ee-96ff-4c17-ae07-9b75e9bec203 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:21.929113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379bb4ee-96ff-4c17-ae07-9b75e9bec203 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:21.930590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:22.015448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 795/1577] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:32:21 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:22.016111 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a67564-6635-416a-bab0-b9ecca7f1eb5 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:22.051201 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 783/1578] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:32:21 2022] POST /identity/v3/projects => generated 389 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:22.051201 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379bb4ee-96ff-4c17-ae07-9b75e9bec203 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:22.128944 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a67564-6635-416a-bab0-b9ecca7f1eb5 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:22.149769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379bb4ee-96ff-4c17-ae07-9b75e9bec203 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:22.158400 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da0d75-1c34-49e9-99e2-43cd06545028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:22.159424 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da0d75-1c34-49e9-99e2-43cd06545028 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:22.159712 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da0d75-1c34-49e9-99e2-43cd06545028 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:22.159998 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da0d75-1c34-49e9-99e2-43cd06545028 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:22.160672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da0d75-1c34-49e9-99e2-43cd06545028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:22.162321 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da0d75-1c34-49e9-99e2-43cd06545028 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:22.175154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 784/1579] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:22.176109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da0d75-1c34-49e9-99e2-43cd06545028 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:22.190763 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da0d75-1c34-49e9-99e2-43cd06545028 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:22.197069 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a07943-0944-4906-9456-3aeff98e2fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:22.198134 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a07943-0944-4906-9456-3aeff98e2fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:22.198632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a07943-0944-4906-9456-3aeff98e2fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:22.198939 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a07943-0944-4906-9456-3aeff98e2fc8 admin admin] PATH_INFO: `/v3/projects/1070aeddb0d544f4b3b4c1e41979944d/users/a145e63a44db43acafc92bb2a238f463/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:22.211886 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7089e37-fbf8-4a7b-9285-7fdb00440e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:22.212610 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7089e37-fbf8-4a7b-9285-7fdb00440e43 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:22.212893 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7089e37-fbf8-4a7b-9285-7fdb00440e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:22.213185 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7089e37-fbf8-4a7b-9285-7fdb00440e43 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:22.213930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7089e37-fbf8-4a7b-9285-7fdb00440e43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:22.216668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7089e37-fbf8-4a7b-9285-7fdb00440e43 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:22.222759 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a07943-0944-4906-9456-3aeff98e2fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1070aeddb0d544f4b3b4c1e41979944d, user_id=a145e63a44db43acafc92bb2a238f463, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:22.224115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: WARNING py.warnings [None req-a2a07943-0944-4906-9456-3aeff98e2fc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:32:22.224115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: warnings.warn(msg) Sep 07 13:32:22.224115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: Sep 07 13:32:22.226811 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a07943-0944-4906-9456-3aeff98e2fc8 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:22.351170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 785/1580] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:22 2022] PUT /identity/v3/projects/1070aeddb0d544f4b3b4c1e41979944d/users/a145e63a44db43acafc92bb2a238f463/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:22.352164 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a07943-0944-4906-9456-3aeff98e2fc8 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:22.397656 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 796/1581] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:32:22 2022] POST /identity/v3/projects => generated 405 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:22.397656 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7089e37-fbf8-4a7b-9285-7fdb00440e43 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:22.433884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a07943-0944-4906-9456-3aeff98e2fc8 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:22.440567 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:22.515195 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7089e37-fbf8-4a7b-9285-7fdb00440e43 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:22.522504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd755f2a-3727-4584-864c-a794285113b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:22.523469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd755f2a-3727-4584-864c-a794285113b1 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:22.523786 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd755f2a-3727-4584-864c-a794285113b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:22.524081 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd755f2a-3727-4584-864c-a794285113b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:22.525451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd755f2a-3727-4584-864c-a794285113b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:22.527825 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd755f2a-3727-4584-864c-a794285113b1 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:22.547965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f714412-4035-448f-b667-0e488ac73e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:22.548607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f714412-4035-448f-b667-0e488ac73e1c admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:22.548889 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f714412-4035-448f-b667-0e488ac73e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:22.549168 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f714412-4035-448f-b667-0e488ac73e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:22.552673 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 797/1582] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:22.552938 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f714412-4035-448f-b667-0e488ac73e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:22.553061 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd755f2a-3727-4584-864c-a794285113b1 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:22.553194 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f714412-4035-448f-b667-0e488ac73e1c admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:22.568434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 786/1583] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:22.569421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f714412-4035-448f-b667-0e488ac73e1c admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:22.570750 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd755f2a-3727-4584-864c-a794285113b1 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:22.579985 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f36e32-e782-4cc9-a353-958a4e1005bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:22.580682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:22.581012 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f36e32-e782-4cc9-a353-958a4e1005bc admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:22.581582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f36e32-e782-4cc9-a353-958a4e1005bc admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:22.582029 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f36e32-e782-4cc9-a353-958a4e1005bc admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:22.582633 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f36e32-e782-4cc9-a353-958a4e1005bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:22.585919 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f36e32-e782-4cc9-a353-958a4e1005bc admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:22.586816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f714412-4035-448f-b667-0e488ac73e1c admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:22.593741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e337c-58c2-4964-a101-fa812d97a77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:22.598065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e337c-58c2-4964-a101-fa812d97a77a admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:22.598065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e337c-58c2-4964-a101-fa812d97a77a admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:22.598065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e337c-58c2-4964-a101-fa812d97a77a admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:22.598065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e337c-58c2-4964-a101-fa812d97a77a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:22.599209 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e337c-58c2-4964-a101-fa812d97a77a admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:22.611222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 798/1584] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:22.611222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f36e32-e782-4cc9-a353-958a4e1005bc admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:22.629291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f36e32-e782-4cc9-a353-958a4e1005bc admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:22.640128 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb61ba5-85fe-42df-9bc3-ffbe33247cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:22.640860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb61ba5-85fe-42df-9bc3-ffbe33247cda admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:22.641971 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb61ba5-85fe-42df-9bc3-ffbe33247cda admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:22.641971 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb61ba5-85fe-42df-9bc3-ffbe33247cda admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:22.642332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb61ba5-85fe-42df-9bc3-ffbe33247cda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:22.646219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb61ba5-85fe-42df-9bc3-ffbe33247cda admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:22.734093 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 787/1585] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:32:22 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:22.734881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e337c-58c2-4964-a101-fa812d97a77a admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:22.745671 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e337c-58c2-4964-a101-fa812d97a77a admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:22.751541 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63f53b-f531-4b37-988f-2bdca83b9b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:22.752377 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63f53b-f531-4b37-988f-2bdca83b9b5d admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:22.752704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63f53b-f531-4b37-988f-2bdca83b9b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:22.752994 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63f53b-f531-4b37-988f-2bdca83b9b5d admin admin] PATH_INFO: `/v3/projects/e0e892aa9df049c387360b67c5e2e065/users/7e7db29f64934026abbe96d5cb5baf25/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:22.774657 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63f53b-f531-4b37-988f-2bdca83b9b5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0e892aa9df049c387360b67c5e2e065, user_id=7e7db29f64934026abbe96d5cb5baf25, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:22.777027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 799/1586] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:32:22 2022] POST /identity/v3/users => generated 295 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:22.777796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63f53b-f531-4b37-988f-2bdca83b9b5d admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:22.778926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e4392-208f-4f5d-8d67-9f00d6523132 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:22.779452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e4392-208f-4f5d-8d67-9f00d6523132 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:22.779932 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e4392-208f-4f5d-8d67-9f00d6523132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:22.807614 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-f77e4392-208f-4f5d-8d67-9f00d6523132 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:22.808868 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f77e4392-208f-4f5d-8d67-9f00d6523132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:22.817636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 800/1587] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:22.819052 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e4392-208f-4f5d-8d67-9f00d6523132 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:22.837630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e4392-208f-4f5d-8d67-9f00d6523132 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:22.845046 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea418f-d266-4efa-957d-04ffa8b53bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:22.845964 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea418f-d266-4efa-957d-04ffa8b53bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:22.846385 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea418f-d266-4efa-957d-04ffa8b53bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:22.846850 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea418f-d266-4efa-957d-04ffa8b53bf1 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:22.848000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea418f-d266-4efa-957d-04ffa8b53bf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:22.850142 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea418f-d266-4efa-957d-04ffa8b53bf1 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:22.862963 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 788/1588] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:22 2022] PUT /identity/v3/projects/e0e892aa9df049c387360b67c5e2e065/users/7e7db29f64934026abbe96d5cb5baf25/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:22.863885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63f53b-f531-4b37-988f-2bdca83b9b5d admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:22.948116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63f53b-f531-4b37-988f-2bdca83b9b5d admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:22.953934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846cc5d3-26b0-45d6-8efa-914afc3bc98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:22.954999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cc5d3-26b0-45d6-8efa-914afc3bc98d admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:22.955489 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cc5d3-26b0-45d6-8efa-914afc3bc98d admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:22.955858 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cc5d3-26b0-45d6-8efa-914afc3bc98d admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:22.956649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846cc5d3-26b0-45d6-8efa-914afc3bc98d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:22.958393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846cc5d3-26b0-45d6-8efa-914afc3bc98d admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:22.977150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 789/1589] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:22.977150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846cc5d3-26b0-45d6-8efa-914afc3bc98d admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:22.985087 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 801/1590] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:32:22 2022] POST /identity/v3/users => generated 303 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:22.985087 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea418f-d266-4efa-957d-04ffa8b53bf1 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:22.991183 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846cc5d3-26b0-45d6-8efa-914afc3bc98d admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:22.998294 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea418f-d266-4efa-957d-04ffa8b53bf1 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:23.000967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96695575-d688-4168-ac1d-fa9afa28ad15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:23.002028 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96695575-d688-4168-ac1d-fa9afa28ad15 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.002746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96695575-d688-4168-ac1d-fa9afa28ad15 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.003057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96695575-d688-4168-ac1d-fa9afa28ad15 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.004196 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96695575-d688-4168-ac1d-fa9afa28ad15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:23.004695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f56c19-d828-40d0-8422-e211447ee690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:23.005903 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f56c19-d828-40d0-8422-e211447ee690 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.006357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f56c19-d828-40d0-8422-e211447ee690 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.006638 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f56c19-d828-40d0-8422-e211447ee690 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.007359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f56c19-d828-40d0-8422-e211447ee690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:23.013133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96695575-d688-4168-ac1d-fa9afa28ad15 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:23.013320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f56c19-d828-40d0-8422-e211447ee690 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:23.032449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 790/1591] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:23.032449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96695575-d688-4168-ac1d-fa9afa28ad15 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:23.032746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 802/1592] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:23.032746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f56c19-d828-40d0-8422-e211447ee690 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:23.047982 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f56c19-d828-40d0-8422-e211447ee690 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:23.048969 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96695575-d688-4168-ac1d-fa9afa28ad15 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:23.053763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcdf27-c791-403b-9c77-efa9f28f707f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:23.054626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcdf27-c791-403b-9c77-efa9f28f707f admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.054916 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcdf27-c791-403b-9c77-efa9f28f707f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.055263 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcdf27-c791-403b-9c77-efa9f28f707f admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.055927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bcdf27-c791-403b-9c77-efa9f28f707f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:23.056352 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95981284-926c-4a37-a7f2-441d43048c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:23.057403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95981284-926c-4a37-a7f2-441d43048c8e admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.057608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bcdf27-c791-403b-9c77-efa9f28f707f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:23.057752 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95981284-926c-4a37-a7f2-441d43048c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.058040 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95981284-926c-4a37-a7f2-441d43048c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.058719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95981284-926c-4a37-a7f2-441d43048c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:23.061267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95981284-926c-4a37-a7f2-441d43048c8e admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:23.075519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 791/1593] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:23.075519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95981284-926c-4a37-a7f2-441d43048c8e admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:23.076294 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 803/1594] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:23.077963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5339f2d-ac66-4a3c-abd9-4696db134157 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.078427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5339f2d-ac66-4a3c-abd9-4696db134157 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.078705 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5339f2d-ac66-4a3c-abd9-4696db134157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.103791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-c5339f2d-ac66-4a3c-abd9-4696db134157 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:23.106080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c5339f2d-ac66-4a3c-abd9-4696db134157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:23.121097 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 804/1595] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:23.121097 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5339f2d-ac66-4a3c-abd9-4696db134157 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:23.136413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5339f2d-ac66-4a3c-abd9-4696db134157 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:23.143827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae83d3-50bf-46a6-9097-0b479a73097b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:23.144777 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae83d3-50bf-46a6-9097-0b479a73097b admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.145253 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae83d3-50bf-46a6-9097-0b479a73097b admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.145872 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae83d3-50bf-46a6-9097-0b479a73097b admin admin] PATH_INFO: `/v3/projects/28c787bb7f894a8d950d27a50f1f6284/users/a93907c2bdc84d39a331747fffa8b6a8/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.152630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95981284-926c-4a37-a7f2-441d43048c8e admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:23.158679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018aea88-a542-43d9-bbd0-39786fef0a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:23.158679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aea88-a542-43d9-bbd0-39786fef0a19 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.162323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aea88-a542-43d9-bbd0-39786fef0a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.162323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aea88-a542-43d9-bbd0-39786fef0a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.169818 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae83d3-50bf-46a6-9097-0b479a73097b admin admin] RBAC: Authorizing `identity:create_grant(project_id=28c787bb7f894a8d950d27a50f1f6284, user_id=a93907c2bdc84d39a331747fffa8b6a8, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:23.170838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018aea88-a542-43d9-bbd0-39786fef0a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:23.172662 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018aea88-a542-43d9-bbd0-39786fef0a19 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:23.173415 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae83d3-50bf-46a6-9097-0b479a73097b admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:23.188707 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 792/1596] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:23.190687 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018aea88-a542-43d9-bbd0-39786fef0a19 service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:23.202188 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018aea88-a542-43d9-bbd0-39786fef0a19 service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:23.207728 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff2940-b84b-450d-951d-4ec72be3dcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:23.208364 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff2940-b84b-450d-951d-4ec72be3dcbc admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.208651 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff2940-b84b-450d-951d-4ec72be3dcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.208925 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff2940-b84b-450d-951d-4ec72be3dcbc admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.209577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff2940-b84b-450d-951d-4ec72be3dcbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:23.211294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff2940-b84b-450d-951d-4ec72be3dcbc admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:23.222694 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 793/1597] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:23.223717 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff2940-b84b-450d-951d-4ec72be3dcbc admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:23.234469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff2940-b84b-450d-951d-4ec72be3dcbc admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:23.239600 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4fe59-f53d-45c9-9eba-e273efcd7c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:23.240220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4fe59-f53d-45c9-9eba-e273efcd7c23 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.240501 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4fe59-f53d-45c9-9eba-e273efcd7c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.240777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4fe59-f53d-45c9-9eba-e273efcd7c23 admin admin] PATH_INFO: `/v3/projects/5db02cd9acdc4dc9b42a6b1ee5cc2f23/users/80341b19855e496b96b2948b916db613/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.259242 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4fe59-f53d-45c9-9eba-e273efcd7c23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db02cd9acdc4dc9b42a6b1ee5cc2f23, user_id=80341b19855e496b96b2948b916db613, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:23.261574 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4fe59-f53d-45c9-9eba-e273efcd7c23 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:23.271785 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 805/1598] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:23 2022] PUT /identity/v3/projects/28c787bb7f894a8d950d27a50f1f6284/users/a93907c2bdc84d39a331747fffa8b6a8/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:23.272687 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae83d3-50bf-46a6-9097-0b479a73097b admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:23.351845 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae83d3-50bf-46a6-9097-0b479a73097b admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:23.357907 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8e8fd-7b56-4866-b2b9-e6ca3b5a8d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:23.358921 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8e8fd-7b56-4866-b2b9-e6ca3b5a8d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.359331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8e8fd-7b56-4866-b2b9-e6ca3b5a8d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.359542 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8e8fd-7b56-4866-b2b9-e6ca3b5a8d62 admin admin] PATH_INFO: `/v3/projects/f183fd3d9d744bfb8626e87314fd992e/users/1972b789e5064088a2a819d1431bec4e/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.380282 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8e8fd-7b56-4866-b2b9-e6ca3b5a8d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f183fd3d9d744bfb8626e87314fd992e, user_id=1972b789e5064088a2a819d1431bec4e, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:23.383338 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8e8fd-7b56-4866-b2b9-e6ca3b5a8d62 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:23.387570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:23.391461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 794/1599] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:23 2022] PUT /identity/v3/projects/5db02cd9acdc4dc9b42a6b1ee5cc2f23/users/80341b19855e496b96b2948b916db613/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:23.393032 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4fe59-f53d-45c9-9eba-e273efcd7c23 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:23.402781 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:23.483065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4fe59-f53d-45c9-9eba-e273efcd7c23 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:23.490361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11b2da-510b-41cb-97f5-aede8b87d6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:23.491019 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11b2da-510b-41cb-97f5-aede8b87d6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.491410 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11b2da-510b-41cb-97f5-aede8b87d6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.491871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11b2da-510b-41cb-97f5-aede8b87d6a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.492739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab11b2da-510b-41cb-97f5-aede8b87d6a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:23.494580 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab11b2da-510b-41cb-97f5-aede8b87d6a0 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:23.508124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 795/1600] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:23.509017 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11b2da-510b-41cb-97f5-aede8b87d6a0 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:23.520401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11b2da-510b-41cb-97f5-aede8b87d6a0 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:23.526211 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea79d46-4bac-4035-8fa4-1946e767af75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:23.527062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea79d46-4bac-4035-8fa4-1946e767af75 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.527416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea79d46-4bac-4035-8fa4-1946e767af75 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.527717 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea79d46-4bac-4035-8fa4-1946e767af75 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.528499 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea79d46-4bac-4035-8fa4-1946e767af75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:23.531031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea79d46-4bac-4035-8fa4-1946e767af75 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:23.543066 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 796/1601] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:23.545925 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea79d46-4bac-4035-8fa4-1946e767af75 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:23.571479 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 806/1602] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:23 2022] PUT /identity/v3/projects/f183fd3d9d744bfb8626e87314fd992e/users/1972b789e5064088a2a819d1431bec4e/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:23.574700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8e8fd-7b56-4866-b2b9-e6ca3b5a8d62 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:23.689894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea79d46-4bac-4035-8fa4-1946e767af75 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:23.703837 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8e8fd-7b56-4866-b2b9-e6ca3b5a8d62 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:23.710793 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6394a4d-e985-4298-b405-c818f4525387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:23.711463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6394a4d-e985-4298-b405-c818f4525387 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.711743 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6394a4d-e985-4298-b405-c818f4525387 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.712022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6394a4d-e985-4298-b405-c818f4525387 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.712734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6394a4d-e985-4298-b405-c818f4525387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:23.714911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6394a4d-e985-4298-b405-c818f4525387 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:23.729039 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 807/1603] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:23.730690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1786d-6c24-48f1-8aa8-ef6347d3fcb2 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.731098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1786d-6c24-48f1-8aa8-ef6347d3fcb2 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.732101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1786d-6c24-48f1-8aa8-ef6347d3fcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.771133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-d9e1786d-6c24-48f1-8aa8-ef6347d3fcb2 None None] MFA Rules not processed for user `a93907c2bdc84d39a331747fffa8b6a8`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:23.773839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d9e1786d-6c24-48f1-8aa8-ef6347d3fcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:23.788639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aff919-22f4-4166-bee1-a5c9dd3de7ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:23.789622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aff919-22f4-4166-bee1-a5c9dd3de7ba service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.790081 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aff919-22f4-4166-bee1-a5c9dd3de7ba service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.790541 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aff919-22f4-4166-bee1-a5c9dd3de7ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.807327 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aff919-22f4-4166-bee1-a5c9dd3de7ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:23.809039 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aff919-22f4-4166-bee1-a5c9dd3de7ba service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:23.822910 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 797/1604] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:23.824589 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c08a2-c8a9-4b30-bd91-8710b7b2ad4b None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.825611 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c08a2-c8a9-4b30-bd91-8710b7b2ad4b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.826047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c08a2-c8a9-4b30-bd91-8710b7b2ad4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.852244 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-247c08a2-c8a9-4b30-bd91-8710b7b2ad4b None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:23.853960 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-247c08a2-c8a9-4b30-bd91-8710b7b2ad4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:23.866217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 798/1605] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:23.868121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c08a2-c8a9-4b30-bd91-8710b7b2ad4b None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:23.882950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c08a2-c8a9-4b30-bd91-8710b7b2ad4b None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:23.888230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 808/1606] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:23.889510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1786d-6c24-48f1-8aa8-ef6347d3fcb2 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:23.890386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d68e82-2883-41c3-8f6c-2a168ff7ba7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:23.891879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d68e82-2883-41c3-8f6c-2a168ff7ba7a admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.892380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d68e82-2883-41c3-8f6c-2a168ff7ba7a admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.892846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d68e82-2883-41c3-8f6c-2a168ff7ba7a admin admin] PATH_INFO: `/v3/projects/6a627a9e151e4e8d8522afbc844c550c/users/32401931d75649018f6b0e55144bb5f2/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.907234 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1786d-6c24-48f1-8aa8-ef6347d3fcb2 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:23.917062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2619b-3fcb-4d0f-a142-9ddb15e04295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:23.920143 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2619b-3fcb-4d0f-a142-9ddb15e04295 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:23.920542 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2619b-3fcb-4d0f-a142-9ddb15e04295 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:23.921034 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2619b-3fcb-4d0f-a142-9ddb15e04295 admin admin] PATH_INFO: `/v3/projects/495aadbbe15c4f18a5018f45e8bab423/users/c5e931aa22344990a0f032159ebc20ad/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:23.922557 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d68e82-2883-41c3-8f6c-2a168ff7ba7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a627a9e151e4e8d8522afbc844c550c, user_id=32401931d75649018f6b0e55144bb5f2, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:23.925651 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d68e82-2883-41c3-8f6c-2a168ff7ba7a admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:23.951550 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2619b-3fcb-4d0f-a142-9ddb15e04295 admin admin] RBAC: Authorizing `identity:create_grant(project_id=495aadbbe15c4f18a5018f45e8bab423, user_id=c5e931aa22344990a0f032159ebc20ad, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:23.957617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2619b-3fcb-4d0f-a142-9ddb15e04295 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:24.012982 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 799/1607] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:23 2022] PUT /identity/v3/projects/6a627a9e151e4e8d8522afbc844c550c/users/32401931d75649018f6b0e55144bb5f2/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:24.015097 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f570a7d-fa2d-490d-b471-60effad9440a None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:24.015426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f570a7d-fa2d-490d-b471-60effad9440a None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:24.015709 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f570a7d-fa2d-490d-b471-60effad9440a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:24.043882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 809/1608] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:23 2022] PUT /identity/v3/projects/495aadbbe15c4f18a5018f45e8bab423/users/c5e931aa22344990a0f032159ebc20ad/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:24.053266 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2619b-3fcb-4d0f-a142-9ddb15e04295 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:24.081001 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-1f570a7d-fa2d-490d-b471-60effad9440a None None] MFA Rules not processed for user `1972b789e5064088a2a819d1431bec4e`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:24.086931 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-1f570a7d-fa2d-490d-b471-60effad9440a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:24.107007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:24.208143 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:24.217983 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2619b-3fcb-4d0f-a142-9ddb15e04295 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:24.229372 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924efdf1-4c8a-4372-906e-2e3e4ecb668c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:24.230804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924efdf1-4c8a-4372-906e-2e3e4ecb668c admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:24.231997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 800/1609] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:24.236208 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924efdf1-4c8a-4372-906e-2e3e4ecb668c admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:24.236208 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924efdf1-4c8a-4372-906e-2e3e4ecb668c admin admin] PATH_INFO: `/v3/projects/3a808904f29044cc8bb01573c860702d/users/cbea600cbfc74c069b457c4bba559287/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:24.239031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f570a7d-fa2d-490d-b471-60effad9440a None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:24.296151 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924efdf1-4c8a-4372-906e-2e3e4ecb668c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a808904f29044cc8bb01573c860702d, user_id=cbea600cbfc74c069b457c4bba559287, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:24.304863 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924efdf1-4c8a-4372-906e-2e3e4ecb668c admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:24.379528 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f570a7d-fa2d-490d-b471-60effad9440a None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:24.389150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583a33a-39f6-4d7d-9443-e1b7b9a6e23c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:24.390154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583a33a-39f6-4d7d-9443-e1b7b9a6e23c service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:24.390636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583a33a-39f6-4d7d-9443-e1b7b9a6e23c service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:24.391050 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583a33a-39f6-4d7d-9443-e1b7b9a6e23c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:24.407986 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583a33a-39f6-4d7d-9443-e1b7b9a6e23c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:24.410010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583a33a-39f6-4d7d-9443-e1b7b9a6e23c service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:24.422963 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 801/1610] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:24.424637 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca63e62-5bd8-446a-b42d-50221521c418 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:24.430125 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca63e62-5bd8-446a-b42d-50221521c418 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:24.430125 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca63e62-5bd8-446a-b42d-50221521c418 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:24.430125 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 802/1611] 10.222.0.62 () {54 vars in 962 bytes} [Wed Sep 7 13:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:32:24.432784 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3025b-4d49-45df-9c12-41c5d9049f2b None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:24.433299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3025b-4d49-45df-9c12-41c5d9049f2b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:24.433746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3025b-4d49-45df-9c12-41c5d9049f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:24.459041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-a5b3025b-4d49-45df-9c12-41c5d9049f2b None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:24.460500 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a5b3025b-4d49-45df-9c12-41c5d9049f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:24.470364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 810/1612] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:24 2022] PUT /identity/v3/projects/3a808904f29044cc8bb01573c860702d/users/cbea600cbfc74c069b457c4bba559287/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:24.472097 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b83a6f-9bc4-4533-a805-2469304a296e None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:24.476591 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b83a6f-9bc4-4533-a805-2469304a296e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:24.476591 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b83a6f-9bc4-4533-a805-2469304a296e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:24.505021 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-89b83a6f-9bc4-4533-a805-2469304a296e None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:24.506318 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-89b83a6f-9bc4-4533-a805-2469304a296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:24.598394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 803/1613] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:24.600039 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3025b-4d49-45df-9c12-41c5d9049f2b None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:24.620355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3025b-4d49-45df-9c12-41c5d9049f2b None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:24.634568 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 811/1614] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:24.634879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775fe0c2-22f6-4992-8d34-d17316aec9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:24.634879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fe0c2-22f6-4992-8d34-d17316aec9f4 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:24.635461 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9f292-dc18-4b64-ad71-23ecf4128ac4 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:24.635632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fe0c2-22f6-4992-8d34-d17316aec9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:24.635808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9f292-dc18-4b64-ad71-23ecf4128ac4 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:24.635975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fe0c2-22f6-4992-8d34-d17316aec9f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:24.636142 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9f292-dc18-4b64-ad71-23ecf4128ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:24.636315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775fe0c2-22f6-4992-8d34-d17316aec9f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:24.638720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775fe0c2-22f6-4992-8d34-d17316aec9f4 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:24.707208 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-eff9f292-dc18-4b64-ad71-23ecf4128ac4 None None] MFA Rules not processed for user `97c363046db444cbb3d4346066f9ad42`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:24.708863 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-eff9f292-dc18-4b64-ad71-23ecf4128ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:24.767425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 804/1615] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:32:24 2022] POST /identity/v3/projects => generated 365 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:24.773147 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b532bd-e096-4e0a-92d5-c6cf821478a2 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:24.773607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b532bd-e096-4e0a-92d5-c6cf821478a2 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:24.776389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b532bd-e096-4e0a-92d5-c6cf821478a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:24.824783 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-e5b532bd-e096-4e0a-92d5-c6cf821478a2 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:24.827726 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e5b532bd-e096-4e0a-92d5-c6cf821478a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:24.886329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:24.895601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 812/1616] 10.222.0.62 () {58 vars in 1059 bytes} [Wed Sep 7 13:32:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:24.897681 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9f292-dc18-4b64-ad71-23ecf4128ac4 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:24.977262 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 805/1617] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:24.979253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b532bd-e096-4e0a-92d5-c6cf821478a2 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:25.357400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9f292-dc18-4b64-ad71-23ecf4128ac4 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:25.357400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:25.396869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b532bd-e096-4e0a-92d5-c6cf821478a2 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:25.396869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1bcbb-9c83-4797-95f3-0a760cc5ac3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:25.396869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1bcbb-9c83-4797-95f3-0a760cc5ac3d service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:25.396869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1bcbb-9c83-4797-95f3-0a760cc5ac3d service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:25.398006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1bcbb-9c83-4797-95f3-0a760cc5ac3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:25.398637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4ed49-1eac-44d9-9344-012bb2e63e01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:25.398637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4ed49-1eac-44d9-9344-012bb2e63e01 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:25.398637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4ed49-1eac-44d9-9344-012bb2e63e01 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:25.398637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4ed49-1eac-44d9-9344-012bb2e63e01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:25.399959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1bcbb-9c83-4797-95f3-0a760cc5ac3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:25.399959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1bcbb-9c83-4797-95f3-0a760cc5ac3d service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:25.400422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4ed49-1eac-44d9-9344-012bb2e63e01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:25.400422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4ed49-1eac-44d9-9344-012bb2e63e01 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:25.400422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 813/1618] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:25.400422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4ed49-1eac-44d9-9344-012bb2e63e01 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:25.401013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 806/1619] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:25.401013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1bcbb-9c83-4797-95f3-0a760cc5ac3d service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:25.401168 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4ed49-1eac-44d9-9344-012bb2e63e01 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:25.401168 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64bc87-7fab-45ba-996a-e001f13c1b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:25.401168 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64bc87-7fab-45ba-996a-e001f13c1b49 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:25.401168 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64bc87-7fab-45ba-996a-e001f13c1b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:25.401476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64bc87-7fab-45ba-996a-e001f13c1b49 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:25.401476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64bc87-7fab-45ba-996a-e001f13c1b49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:25.401476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64bc87-7fab-45ba-996a-e001f13c1b49 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:25.401660 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1bcbb-9c83-4797-95f3-0a760cc5ac3d service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:25.401660 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3f01c-d26a-46c3-aaf0-ad8eb17c6b9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:25.401660 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3f01c-d26a-46c3-aaf0-ad8eb17c6b9e service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:25.401660 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3f01c-d26a-46c3-aaf0-ad8eb17c6b9e service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:25.401909 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3f01c-d26a-46c3-aaf0-ad8eb17c6b9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:25.429689 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3f01c-d26a-46c3-aaf0-ad8eb17c6b9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:25.431345 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3f01c-d26a-46c3-aaf0-ad8eb17c6b9e service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:25.442231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 807/1620] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:25.443432 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3f01c-d26a-46c3-aaf0-ad8eb17c6b9e service nova] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:25.456697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3f01c-d26a-46c3-aaf0-ad8eb17c6b9e service nova] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:25.460695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d0c64-9357-46ee-b138-59a0a5e8e6cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:25.461409 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d0c64-9357-46ee-b138-59a0a5e8e6cc service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:25.461697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d0c64-9357-46ee-b138-59a0a5e8e6cc service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:25.461971 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d0c64-9357-46ee-b138-59a0a5e8e6cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:25.471139 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:25.474745 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d0c64-9357-46ee-b138-59a0a5e8e6cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:25.477682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d0c64-9357-46ee-b138-59a0a5e8e6cc service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:25.493478 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 808/1621] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:25.588698 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 814/1622] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:32:25 2022] POST /identity/v3/users => generated 289 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:25.595100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d0c64-9357-46ee-b138-59a0a5e8e6cc service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:25.617137 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d0c64-9357-46ee-b138-59a0a5e8e6cc service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:25.626590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da21c6a-8424-456c-a7b8-5ef95bac5cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:25.627801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da21c6a-8424-456c-a7b8-5ef95bac5cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:25.628377 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da21c6a-8424-456c-a7b8-5ef95bac5cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:25.628902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da21c6a-8424-456c-a7b8-5ef95bac5cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:25.631832 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da21c6a-8424-456c-a7b8-5ef95bac5cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:25.633811 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da21c6a-8424-456c-a7b8-5ef95bac5cd9 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:25.657901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 809/1623] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:25.665094 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64bc87-7fab-45ba-996a-e001f13c1b49 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:25.690345 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64bc87-7fab-45ba-996a-e001f13c1b49 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:25.697699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20911431-6aa3-42b3-8c15-3e1ebc2afcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:25.702677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20911431-6aa3-42b3-8c15-3e1ebc2afcd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:25.703190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20911431-6aa3-42b3-8c15-3e1ebc2afcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:25.703570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20911431-6aa3-42b3-8c15-3e1ebc2afcd6 admin admin] PATH_INFO: `/v3/projects/e4531b8689e54cee904a7fb2a4df1d19/users/dfa1b8332d0841fe9eff9136736b5844/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:25.736658 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20911431-6aa3-42b3-8c15-3e1ebc2afcd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4531b8689e54cee904a7fb2a4df1d19, user_id=dfa1b8332d0841fe9eff9136736b5844, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:25.741772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20911431-6aa3-42b3-8c15-3e1ebc2afcd6 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:25.896224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 815/1624] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:25 2022] PUT /identity/v3/projects/e4531b8689e54cee904a7fb2a4df1d19/users/dfa1b8332d0841fe9eff9136736b5844/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:25.900703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da21c6a-8424-456c-a7b8-5ef95bac5cd9 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:25.932627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da21c6a-8424-456c-a7b8-5ef95bac5cd9 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:26.122753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedcd63-d623-420b-829e-78765ef086de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:26.122753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedcd63-d623-420b-829e-78765ef086de admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:26.122753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedcd63-d623-420b-829e-78765ef086de admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:26.122753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedcd63-d623-420b-829e-78765ef086de admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:26.123444 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bedcd63-d623-420b-829e-78765ef086de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:26.123444 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bedcd63-d623-420b-829e-78765ef086de admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:26.163907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 810/1625] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:26.166721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20911431-6aa3-42b3-8c15-3e1ebc2afcd6 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:26.187481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20911431-6aa3-42b3-8c15-3e1ebc2afcd6 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:26.203188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5b9ff-2065-4c66-84dd-e1e4d4b16f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:26.204318 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5b9ff-2065-4c66-84dd-e1e4d4b16f90 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:26.216228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5b9ff-2065-4c66-84dd-e1e4d4b16f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:26.216228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5b9ff-2065-4c66-84dd-e1e4d4b16f90 admin admin] PATH_INFO: `/v3/projects/e4531b8689e54cee904a7fb2a4df1d19/users/dfa1b8332d0841fe9eff9136736b5844/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:26.216228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5b9ff-2065-4c66-84dd-e1e4d4b16f90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4531b8689e54cee904a7fb2a4df1d19, user_id=dfa1b8332d0841fe9eff9136736b5844, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:26.223183 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5b9ff-2065-4c66-84dd-e1e4d4b16f90 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:26.251345 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 816/1626] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:26 2022] PUT /identity/v3/projects/e4531b8689e54cee904a7fb2a4df1d19/users/dfa1b8332d0841fe9eff9136736b5844/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:26.270365 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169ea3c-8f0c-4b9b-bc2a-3feabe094cfb None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:26.274073 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169ea3c-8f0c-4b9b-bc2a-3feabe094cfb None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:26.274073 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169ea3c-8f0c-4b9b-bc2a-3feabe094cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:26.387464 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-e169ea3c-8f0c-4b9b-bc2a-3feabe094cfb None None] MFA Rules not processed for user `dfa1b8332d0841fe9eff9136736b5844`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:26.387637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64131ef1-acd8-4d4d-a352-9a129d1fc0e9 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:26.387637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64131ef1-acd8-4d4d-a352-9a129d1fc0e9 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:26.387637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64131ef1-acd8-4d4d-a352-9a129d1fc0e9 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:26.387637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 817/1627] 10.222.0.62 () {54 vars in 962 bytes} [Wed Sep 7 13:32:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:32:26.390128 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e169ea3c-8f0c-4b9b-bc2a-3feabe094cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:26.396607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc61035-96d1-4f36-bedb-20449c897b03 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:26.396890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc61035-96d1-4f36-bedb-20449c897b03 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:26.397187 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc61035-96d1-4f36-bedb-20449c897b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:26.433134 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-6fc61035-96d1-4f36-bedb-20449c897b03 None None] MFA Rules not processed for user `97c363046db444cbb3d4346066f9ad42`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:26.435097 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-6fc61035-96d1-4f36-bedb-20449c897b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:26.595487 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 818/1628] 10.222.0.62 () {58 vars in 1059 bytes} [Wed Sep 7 13:32:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:32:26.625958 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 811/1629] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:32:26.626043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d2778-aaad-4514-880a-d136ca722f07 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:26.626043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d2778-aaad-4514-880a-d136ca722f07 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:26.626043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d2778-aaad-4514-880a-d136ca722f07 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:26.632885 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 819/1630] 10.222.0.62 () {54 vars in 912 bytes} [Wed Sep 7 13:32:26 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:32:26.646125 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b0a02-7b34-4600-b3d9-38691da46ca8 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:26.646804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b0a02-7b34-4600-b3d9-38691da46ca8 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:26.655580 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b0a02-7b34-4600-b3d9-38691da46ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:26.655822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d2778-aaad-4514-880a-d136ca722f07 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:26.700335 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-ef2b0a02-7b34-4600-b3d9-38691da46ca8 None None] MFA Rules not processed for user `97c363046db444cbb3d4346066f9ad42`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:26.700335 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ef2b0a02-7b34-4600-b3d9-38691da46ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:26.727480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 812/1631] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 7 13:32:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:26.741613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b0a02-7b34-4600-b3d9-38691da46ca8 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:26.820889 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d2778-aaad-4514-880a-d136ca722f07 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:26.827403 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451678c-39e2-4844-b7de-1a081baf2318 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:26.828180 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451678c-39e2-4844-b7de-1a081baf2318 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:26.828470 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451678c-39e2-4844-b7de-1a081baf2318 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:26.828780 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451678c-39e2-4844-b7de-1a081baf2318 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:26.847021 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451678c-39e2-4844-b7de-1a081baf2318 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:26.868033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451678c-39e2-4844-b7de-1a081baf2318 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:26.868789 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b0a02-7b34-4600-b3d9-38691da46ca8 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:26.874344 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:26.881838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281b31b-9ad0-4df3-b735-392753d5f357 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:26.883061 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281b31b-9ad0-4df3-b735-392753d5f357 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:26.884984 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281b31b-9ad0-4df3-b735-392753d5f357 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:26.885666 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281b31b-9ad0-4df3-b735-392753d5f357 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:26.891108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 820/1632] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:32:26.925117 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281b31b-9ad0-4df3-b735-392753d5f357 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:26.928782 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281b31b-9ad0-4df3-b735-392753d5f357 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:26.966128 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 813/1633] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:27.254091 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00635b6-e7b4-452b-8948-572a5c7c0adc None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:27.256882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00635b6-e7b4-452b-8948-572a5c7c0adc None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:27.258348 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00635b6-e7b4-452b-8948-572a5c7c0adc None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:27.261855 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 821/1634] 10.222.0.62 () {54 vars in 912 bytes} [Wed Sep 7 13:32:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:32:27.273274 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b61da-eb48-4ee1-984f-cf567aa1ce3b None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:27.274024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b61da-eb48-4ee1-984f-cf567aa1ce3b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:27.274601 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b61da-eb48-4ee1-984f-cf567aa1ce3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:27.299417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:27.347524 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-b03b61da-eb48-4ee1-984f-cf567aa1ce3b None None] MFA Rules not processed for user `97c363046db444cbb3d4346066f9ad42`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:27.352191 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b03b61da-eb48-4ee1-984f-cf567aa1ce3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:27.425733 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 814/1635] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 7 13:32:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:27.425884 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00635b6-e7b4-452b-8948-572a5c7c0adc None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:27.435948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b61da-eb48-4ee1-984f-cf567aa1ce3b None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:27.447213 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00635b6-e7b4-452b-8948-572a5c7c0adc None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:27.462711 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff48fa-75db-43f3-b96d-f219fdbc03a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:27.464512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff48fa-75db-43f3-b96d-f219fdbc03a7 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:27.465132 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff48fa-75db-43f3-b96d-f219fdbc03a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:27.465898 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff48fa-75db-43f3-b96d-f219fdbc03a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:27.466791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b61da-eb48-4ee1-984f-cf567aa1ce3b None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:27.486150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642699b-8a38-4e75-b533-3ea1bf4598c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:27.486871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642699b-8a38-4e75-b533-3ea1bf4598c0 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:27.487334 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642699b-8a38-4e75-b533-3ea1bf4598c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:27.487612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642699b-8a38-4e75-b533-3ea1bf4598c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:27.515690 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f642699b-8a38-4e75-b533-3ea1bf4598c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:27.517303 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f642699b-8a38-4e75-b533-3ea1bf4598c0 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:27.550178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 815/1636] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:32:27.633417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff48fa-75db-43f3-b96d-f219fdbc03a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:27.635369 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff48fa-75db-43f3-b96d-f219fdbc03a7 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:27.657141 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 822/1637] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:27.772811 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642699b-8a38-4e75-b533-3ea1bf4598c0 service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:27.947279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642699b-8a38-4e75-b533-3ea1bf4598c0 service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:27.964869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439e6e2-f62e-4330-87ad-495294d0587e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:27.966321 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439e6e2-f62e-4330-87ad-495294d0587e admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:27.966787 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439e6e2-f62e-4330-87ad-495294d0587e admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:27.967193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439e6e2-f62e-4330-87ad-495294d0587e admin admin] PATH_INFO: `/v3/users/a93907c2bdc84d39a331747fffa8b6a8` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:27.970358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439e6e2-f62e-4330-87ad-495294d0587e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a93907c2bdc84d39a331747fffa8b6a8)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:27.971667 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: WARNING py.warnings [None req-6439e6e2-f62e-4330-87ad-495294d0587e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:32:27.971667 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: warnings.warn(msg) Sep 07 13:32:27.971667 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: Sep 07 13:32:27.973112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439e6e2-f62e-4330-87ad-495294d0587e admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:28.315883 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-6439e6e2-f62e-4330-87ad-495294d0587e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a93907c2bdc84d39a331747fffa8b6a8'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:28.361439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-6439e6e2-f62e-4330-87ad-495294d0587e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a93907c2bdc84d39a331747fffa8b6a8'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:28.362603 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-6439e6e2-f62e-4330-87ad-495294d0587e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a93907c2bdc84d39a331747fffa8b6a8'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:28.388639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-6439e6e2-f62e-4330-87ad-495294d0587e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a93907c2bdc84d39a331747fffa8b6a8'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:28.390248 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-6439e6e2-f62e-4330-87ad-495294d0587e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a93907c2bdc84d39a331747fffa8b6a8'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:28.614664 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 816/1638] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:32:27 2022] DELETE /identity/v3/users/a93907c2bdc84d39a331747fffa8b6a8 => generated 0 bytes in 844 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:28.616384 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7db06-d908-4a60-a012-358c57ba9973 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:28.617032 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7db06-d908-4a60-a012-358c57ba9973 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:28.617203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7db06-d908-4a60-a012-358c57ba9973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:28.679934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-8af7db06-d908-4a60-a012-358c57ba9973 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:28.681351 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8af7db06-d908-4a60-a012-358c57ba9973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:28.868684 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 823/1639] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:32:29.123965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439e6e2-f62e-4330-87ad-495294d0587e admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:29.126645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6439e6e2-f62e-4330-87ad-495294d0587e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:29.350026 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439e6e2-f62e-4330-87ad-495294d0587e admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:29.371653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b8875-74a2-4d16-a2bd-6a969285fda7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:29.373527 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b8875-74a2-4d16-a2bd-6a969285fda7 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:29.374371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b8875-74a2-4d16-a2bd-6a969285fda7 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:29.375075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b8875-74a2-4d16-a2bd-6a969285fda7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:29.432840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b8875-74a2-4d16-a2bd-6a969285fda7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:29.438263 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b8875-74a2-4d16-a2bd-6a969285fda7 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:29.464328 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 817/1640] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:30.056009 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa526a97-7cc9-4cf2-9de9-5bbe582c6e2b None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:30.057185 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa526a97-7cc9-4cf2-9de9-5bbe582c6e2b None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:30.061619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa526a97-7cc9-4cf2-9de9-5bbe582c6e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:30.180235 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-aa526a97-7cc9-4cf2-9de9-5bbe582c6e2b None None] MFA Rules not processed for user `cbea600cbfc74c069b457c4bba559287`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:30.181978 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-aa526a97-7cc9-4cf2-9de9-5bbe582c6e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:30.417389 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 824/1641] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:30 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:30.425018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b8875-74a2-4d16-a2bd-6a969285fda7 service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:30.427206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b82b8875-74a2-4d16-a2bd-6a969285fda7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:30.452237 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:30.471994 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b8875-74a2-4d16-a2bd-6a969285fda7 service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:30.500062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaa4ec-f2d5-4fb1-bed1-9e3de9d18bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:30.502875 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaa4ec-f2d5-4fb1-bed1-9e3de9d18bdd admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:30.506415 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaa4ec-f2d5-4fb1-bed1-9e3de9d18bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:30.507317 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaa4ec-f2d5-4fb1-bed1-9e3de9d18bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:30.508856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addaa4ec-f2d5-4fb1-bed1-9e3de9d18bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:30.519900 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addaa4ec-f2d5-4fb1-bed1-9e3de9d18bdd admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:30.736192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 818/1642] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:32:30 2022] POST /identity/v3/projects => generated 403 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:30.742684 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa526a97-7cc9-4cf2-9de9-5bbe582c6e2b None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:30.763412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa526a97-7cc9-4cf2-9de9-5bbe582c6e2b None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:30.817845 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:30.900366 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505609b7-7dda-4290-9295-0a14adbee61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:30.901972 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505609b7-7dda-4290-9295-0a14adbee61a admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:30.902642 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505609b7-7dda-4290-9295-0a14adbee61a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:30.903516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505609b7-7dda-4290-9295-0a14adbee61a admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:30.905570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505609b7-7dda-4290-9295-0a14adbee61a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:30.909727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505609b7-7dda-4290-9295-0a14adbee61a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:31.054859 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 825/1643] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:32:30 2022] POST /identity/v3/users => generated 308 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:31.059590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaa4ec-f2d5-4fb1-bed1-9e3de9d18bdd admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:31.081269 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaa4ec-f2d5-4fb1-bed1-9e3de9d18bdd admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:31.091270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8c2f2-09e3-4cb4-b02d-4ea86768d0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:31.092533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8c2f2-09e3-4cb4-b02d-4ea86768d0ef admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:31.093217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8c2f2-09e3-4cb4-b02d-4ea86768d0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:31.093803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8c2f2-09e3-4cb4-b02d-4ea86768d0ef admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:31.095234 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae8c2f2-09e3-4cb4-b02d-4ea86768d0ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:31.098932 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae8c2f2-09e3-4cb4-b02d-4ea86768d0ef admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:31.127617 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 819/1644] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:31.134894 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505609b7-7dda-4290-9295-0a14adbee61a admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:31.158512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505609b7-7dda-4290-9295-0a14adbee61a admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:31.173496 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e73f6f-0207-414b-95b7-46d9825eaf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:31.177495 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e73f6f-0207-414b-95b7-46d9825eaf0e admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:31.178144 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e73f6f-0207-414b-95b7-46d9825eaf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:31.178868 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e73f6f-0207-414b-95b7-46d9825eaf0e admin admin] PATH_INFO: `/v3/projects/3badd2f1a6444efd97043e547bc60930/users/7f8c7dc2115b44128d800086dee7cfe4/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:31.215825 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e73f6f-0207-414b-95b7-46d9825eaf0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3badd2f1a6444efd97043e547bc60930, user_id=7f8c7dc2115b44128d800086dee7cfe4, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:31.219881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e73f6f-0207-414b-95b7-46d9825eaf0e admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:31.379814 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 826/1645] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:31 2022] PUT /identity/v3/projects/3badd2f1a6444efd97043e547bc60930/users/7f8c7dc2115b44128d800086dee7cfe4/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:31.384754 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8c2f2-09e3-4cb4-b02d-4ea86768d0ef admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:31.429946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8c2f2-09e3-4cb4-b02d-4ea86768d0ef admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:31.687484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c092a-8f39-489e-a4bd-78c410dd4305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:31.688980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c092a-8f39-489e-a4bd-78c410dd4305 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:31.689606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c092a-8f39-489e-a4bd-78c410dd4305 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:31.689793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c092a-8f39-489e-a4bd-78c410dd4305 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:31.693173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c092a-8f39-489e-a4bd-78c410dd4305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:31.698524 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c092a-8f39-489e-a4bd-78c410dd4305 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:31.739233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 820/1646] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:32:31.741789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e73f6f-0207-414b-95b7-46d9825eaf0e admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:31.772486 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e73f6f-0207-414b-95b7-46d9825eaf0e admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:31.789635 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186a6e1-2993-4966-8df8-f8238d5ad9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:31.789635 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186a6e1-2993-4966-8df8-f8238d5ad9f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:31.789635 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186a6e1-2993-4966-8df8-f8238d5ad9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:31.790162 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186a6e1-2993-4966-8df8-f8238d5ad9f1 admin admin] PATH_INFO: `/v3/projects/3badd2f1a6444efd97043e547bc60930/users/7f8c7dc2115b44128d800086dee7cfe4/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:31.794355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186a6e1-2993-4966-8df8-f8238d5ad9f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3badd2f1a6444efd97043e547bc60930, user_id=7f8c7dc2115b44128d800086dee7cfe4, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:31.804805 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186a6e1-2993-4966-8df8-f8238d5ad9f1 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:31.844192 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 827/1647] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:31 2022] PUT /identity/v3/projects/3badd2f1a6444efd97043e547bc60930/users/7f8c7dc2115b44128d800086dee7cfe4/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:32.597343 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c092a-8f39-489e-a4bd-78c410dd4305 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:32.607219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f45c092a-8f39-489e-a4bd-78c410dd4305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:32.641133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c092a-8f39-489e-a4bd-78c410dd4305 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:32.845542 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33638aa-7b6e-4f94-a080-f74531e6af33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:32.846885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33638aa-7b6e-4f94-a080-f74531e6af33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:32.849928 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33638aa-7b6e-4f94-a080-f74531e6af33 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:32.850705 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33638aa-7b6e-4f94-a080-f74531e6af33 admin admin] PATH_INFO: `/v3/projects/28c787bb7f894a8d950d27a50f1f6284` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:32.854712 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33638aa-7b6e-4f94-a080-f74531e6af33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28c787bb7f894a8d950d27a50f1f6284)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:32.859822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33638aa-7b6e-4f94-a080-f74531e6af33 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:33.237911 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f33638aa-7b6e-4f94-a080-f74531e6af33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28c787bb7f894a8d950d27a50f1f6284', 'request_id': 'req-f33638aa-7b6e-4f94-a080-f74531e6af33'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:33.279653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f33638aa-7b6e-4f94-a080-f74531e6af33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28c787bb7f894a8d950d27a50f1f6284', 'request_id': 'req-f33638aa-7b6e-4f94-a080-f74531e6af33'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:33.447373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f33638aa-7b6e-4f94-a080-f74531e6af33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28c787bb7f894a8d950d27a50f1f6284', 'request_id': 'req-f33638aa-7b6e-4f94-a080-f74531e6af33'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:33.495923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:33.520041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f33638aa-7b6e-4f94-a080-f74531e6af33 admin admin] The token cache is being invalidate because project 28c787bb7f894a8d950d27a50f1f6284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:32:33.520726 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f33638aa-7b6e-4f94-a080-f74531e6af33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:33.524194 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f33638aa-7b6e-4f94-a080-f74531e6af33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:33.529351 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 821/1648] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:32:32 2022] DELETE /identity/v3/projects/28c787bb7f894a8d950d27a50f1f6284 => generated 0 bytes in 932 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:33.539966 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d92313-32d8-445f-a479-0fec2bfcba02 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:33.545193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d92313-32d8-445f-a479-0fec2bfcba02 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:33.546059 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d92313-32d8-445f-a479-0fec2bfcba02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:33.629570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-20d92313-32d8-445f-a479-0fec2bfcba02 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:33.634500 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-20d92313-32d8-445f-a479-0fec2bfcba02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:33.989736 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 828/1649] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:34.007280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7680f-079c-4fc7-a2fa-fda869569e4e None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:34.008073 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7680f-079c-4fc7-a2fa-fda869569e4e None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:34.008647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7680f-079c-4fc7-a2fa-fda869569e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:34.126790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-61b7680f-079c-4fc7-a2fa-fda869569e4e None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:34.129110 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-61b7680f-079c-4fc7-a2fa-fda869569e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:34.170869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 822/1650] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:34.179418 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d92313-32d8-445f-a479-0fec2bfcba02 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:34.234355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d92313-32d8-445f-a479-0fec2bfcba02 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:34.261644 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f58ed9-c242-43f1-be45-79010cdfd8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:34.268320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f58ed9-c242-43f1-be45-79010cdfd8ba admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:34.269338 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f58ed9-c242-43f1-be45-79010cdfd8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:34.271806 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f58ed9-c242-43f1-be45-79010cdfd8ba admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:34.280336 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f58ed9-c242-43f1-be45-79010cdfd8ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:32:34.281033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f58ed9-c242-43f1-be45-79010cdfd8ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:34.284803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f58ed9-c242-43f1-be45-79010cdfd8ba admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:34.329462 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 829/1651] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:32:34.330064 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7680f-079c-4fc7-a2fa-fda869569e4e None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:34.361696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7680f-079c-4fc7-a2fa-fda869569e4e None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:34.393504 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c8f05-ad87-41db-8fb2-216c48b1f9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:34.395341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c8f05-ad87-41db-8fb2-216c48b1f9e8 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:34.395842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c8f05-ad87-41db-8fb2-216c48b1f9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:34.398579 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c8f05-ad87-41db-8fb2-216c48b1f9e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:34.402078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c8f05-ad87-41db-8fb2-216c48b1f9e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:34.428986 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c8f05-ad87-41db-8fb2-216c48b1f9e8 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:34.489479 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f58ed9-c242-43f1-be45-79010cdfd8ba admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:34.494781 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e7f58ed9-c242-43f1-be45-79010cdfd8ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:34.548886 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f58ed9-c242-43f1-be45-79010cdfd8ba admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:34.568099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb8ee3-3068-472e-8014-69e1d05b9b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:34.570313 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb8ee3-3068-472e-8014-69e1d05b9b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:34.571661 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb8ee3-3068-472e-8014-69e1d05b9b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:34.572158 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb8ee3-3068-472e-8014-69e1d05b9b4f admin admin] PATH_INFO: `/v3/users/1972b789e5064088a2a819d1431bec4e` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:34.575706 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb8ee3-3068-472e-8014-69e1d05b9b4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1972b789e5064088a2a819d1431bec4e)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:34.581685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb8ee3-3068-472e-8014-69e1d05b9b4f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:34.648552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 823/1652] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:32:34 2022] POST /identity/v3/projects => generated 365 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:32:34.654497 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c8f05-ad87-41db-8fb2-216c48b1f9e8 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:34.909933 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c8f05-ad87-41db-8fb2-216c48b1f9e8 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:34.919793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd0237-9b80-46bc-bdb4-4d1e959315db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:34.921383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd0237-9b80-46bc-bdb4-4d1e959315db admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:34.921958 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd0237-9b80-46bc-bdb4-4d1e959315db admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:34.922377 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd0237-9b80-46bc-bdb4-4d1e959315db admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:34.923649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd0237-9b80-46bc-bdb4-4d1e959315db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:34.929034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd0237-9b80-46bc-bdb4-4d1e959315db admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:34.981452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-93bb8ee3-3068-472e-8014-69e1d05b9b4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1972b789e5064088a2a819d1431bec4e'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:35.088085 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-93bb8ee3-3068-472e-8014-69e1d05b9b4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1972b789e5064088a2a819d1431bec4e'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:35.089044 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-93bb8ee3-3068-472e-8014-69e1d05b9b4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1972b789e5064088a2a819d1431bec4e'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:35.089868 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-93bb8ee3-3068-472e-8014-69e1d05b9b4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1972b789e5064088a2a819d1431bec4e'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:35.118235 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 824/1653] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:32:34 2022] POST /identity/v3/users => generated 283 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:35.120871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd0237-9b80-46bc-bdb4-4d1e959315db admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:35.121814 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:35.131596 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-20fd0237-9b80-46bc-bdb4-4d1e959315db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:35.133706 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-93bb8ee3-3068-472e-8014-69e1d05b9b4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1972b789e5064088a2a819d1431bec4e'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:35.168500 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd0237-9b80-46bc-bdb4-4d1e959315db admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:35.191518 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79c64d-d0a5-4d83-8513-e362f30b7d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:35.192042 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79c64d-d0a5-4d83-8513-e362f30b7d92 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:35.192787 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79c64d-d0a5-4d83-8513-e362f30b7d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:35.193835 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79c64d-d0a5-4d83-8513-e362f30b7d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:35.196725 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 830/1654] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:32:34 2022] DELETE /identity/v3/users/1972b789e5064088a2a819d1431bec4e => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:35.198083 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a79c64d-d0a5-4d83-8513-e362f30b7d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:35.208078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4fec6-3d83-4a29-8ebe-40028dd42b48 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:35.208558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4fec6-3d83-4a29-8ebe-40028dd42b48 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:35.209043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4fec6-3d83-4a29-8ebe-40028dd42b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:35.213634 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a79c64d-d0a5-4d83-8513-e362f30b7d92 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:35.275453 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-1eb4fec6-3d83-4a29-8ebe-40028dd42b48 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:35.281166 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1eb4fec6-3d83-4a29-8ebe-40028dd42b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:35.286454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 825/1655] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:35.293100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79c64d-d0a5-4d83-8513-e362f30b7d92 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:35.334846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79c64d-d0a5-4d83-8513-e362f30b7d92 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:35.460188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 831/1656] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:35.720595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4fec6-3d83-4a29-8ebe-40028dd42b48 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:35.724012 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1eb4fec6-3d83-4a29-8ebe-40028dd42b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:35.769713 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5e902-5ada-4b05-828b-e6752c77eb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:35.771694 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5e902-5ada-4b05-828b-e6752c77eb90 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:35.772175 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5e902-5ada-4b05-828b-e6752c77eb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:35.772652 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5e902-5ada-4b05-828b-e6752c77eb90 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:35.774163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5e902-5ada-4b05-828b-e6752c77eb90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:35.779501 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5e902-5ada-4b05-828b-e6752c77eb90 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:35.804359 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 826/1657] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 512 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:32:35.813765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5e902-5ada-4b05-828b-e6752c77eb90 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:35.890003 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5e902-5ada-4b05-828b-e6752c77eb90 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:35.915580 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cc8f2-4733-4e11-8457-8068cef4e6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:35.917112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cc8f2-4733-4e11-8457-8068cef4e6fa admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:35.917619 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cc8f2-4733-4e11-8457-8068cef4e6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:35.918109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cc8f2-4733-4e11-8457-8068cef4e6fa admin admin] PATH_INFO: `/v3/projects/74213b06d3dd4e5998a78692c52e1970/users/3b5aa7791a7f4632af0bc0e90d12173b/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:35.978773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26cc8f2-4733-4e11-8457-8068cef4e6fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=74213b06d3dd4e5998a78692c52e1970, user_id=3b5aa7791a7f4632af0bc0e90d12173b, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:35.988030 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26cc8f2-4733-4e11-8457-8068cef4e6fa admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:36.119735 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 827/1658] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:35 2022] PUT /identity/v3/projects/74213b06d3dd4e5998a78692c52e1970/users/3b5aa7791a7f4632af0bc0e90d12173b/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:36.145492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692a1b8-2d15-4436-8288-5d4ca59a0067 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:36.146114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692a1b8-2d15-4436-8288-5d4ca59a0067 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:36.146406 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692a1b8-2d15-4436-8288-5d4ca59a0067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:36.185517 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4fec6-3d83-4a29-8ebe-40028dd42b48 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:36.230231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-1692a1b8-2d15-4436-8288-5d4ca59a0067 None None] MFA Rules not processed for user `3b5aa7791a7f4632af0bc0e90d12173b`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:36.232364 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-1692a1b8-2d15-4436-8288-5d4ca59a0067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:36.263648 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:36.522743 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230ed3d-d8db-4dbb-a236-3c153b648d04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:36.523885 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230ed3d-d8db-4dbb-a236-3c153b648d04 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:36.524370 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230ed3d-d8db-4dbb-a236-3c153b648d04 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:36.525079 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230ed3d-d8db-4dbb-a236-3c153b648d04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:36.582456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 828/1659] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:32:36.588088 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692a1b8-2d15-4436-8288-5d4ca59a0067 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:36.797040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3230ed3d-d8db-4dbb-a236-3c153b648d04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:36.800017 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3230ed3d-d8db-4dbb-a236-3c153b648d04 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:36.810243 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692a1b8-2d15-4436-8288-5d4ca59a0067 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:36.820439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7abe73-bde7-4485-8aea-394a0c4e8a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:36.825723 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7abe73-bde7-4485-8aea-394a0c4e8a99 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:36.825723 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7abe73-bde7-4485-8aea-394a0c4e8a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:36.825723 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7abe73-bde7-4485-8aea-394a0c4e8a99 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:36.826799 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7abe73-bde7-4485-8aea-394a0c4e8a99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:36.832077 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7abe73-bde7-4485-8aea-394a0c4e8a99 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:36.849107 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 832/1660] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 1129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:32:37.087187 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 829/1661] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:32:36 2022] POST /identity/v3/projects => generated 365 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:37.088572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230ed3d-d8db-4dbb-a236-3c153b648d04 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:37.118624 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230ed3d-d8db-4dbb-a236-3c153b648d04 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:37.186317 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:37.359090 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e5df1-2c51-4b79-b0f7-3db33c31bc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:37.360089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e5df1-2c51-4b79-b0f7-3db33c31bc8a admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:37.361403 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e5df1-2c51-4b79-b0f7-3db33c31bc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:37.362076 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e5df1-2c51-4b79-b0f7-3db33c31bc8a admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:37.363173 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e5df1-2c51-4b79-b0f7-3db33c31bc8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:37.368123 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e5df1-2c51-4b79-b0f7-3db33c31bc8a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:37.539634 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7abe73-bde7-4485-8aea-394a0c4e8a99 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:37.554163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 833/1662] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:32:37 2022] POST /identity/v3/users => generated 289 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:37.573455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7abe73-bde7-4485-8aea-394a0c4e8a99 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:37.601500 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27540699-7f51-49bf-b9b1-d7e26a5a0fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:37.603193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27540699-7f51-49bf-b9b1-d7e26a5a0fbf admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:37.604049 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27540699-7f51-49bf-b9b1-d7e26a5a0fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:37.604839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27540699-7f51-49bf-b9b1-d7e26a5a0fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:37.607161 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27540699-7f51-49bf-b9b1-d7e26a5a0fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:37.620302 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27540699-7f51-49bf-b9b1-d7e26a5a0fbf admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:37.682445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 830/1663] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:32:37.682855 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e5df1-2c51-4b79-b0f7-3db33c31bc8a admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:37.719303 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e5df1-2c51-4b79-b0f7-3db33c31bc8a admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:37.739197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed5f77-6021-4e32-8416-969be26fe4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:37.740425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed5f77-6021-4e32-8416-969be26fe4a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:37.740914 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed5f77-6021-4e32-8416-969be26fe4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:37.741387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed5f77-6021-4e32-8416-969be26fe4a3 admin admin] PATH_INFO: `/v3/projects/f378efa87933420687bb35fb71d1e5f8/users/cd5efa6dc7f6441687dd5ac7ad19567f/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:37.801810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed5f77-6021-4e32-8416-969be26fe4a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f378efa87933420687bb35fb71d1e5f8, user_id=cd5efa6dc7f6441687dd5ac7ad19567f, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:37.808682 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed5f77-6021-4e32-8416-969be26fe4a3 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:37.955289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 834/1664] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:37 2022] PUT /identity/v3/projects/f378efa87933420687bb35fb71d1e5f8/users/cd5efa6dc7f6441687dd5ac7ad19567f/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:37.961566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27540699-7f51-49bf-b9b1-d7e26a5a0fbf admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:38.001461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27540699-7f51-49bf-b9b1-d7e26a5a0fbf admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:38.321672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dcf2ab-266a-4671-becb-b607099b58e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:38.325264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcf2ab-266a-4671-becb-b607099b58e8 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:38.325264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcf2ab-266a-4671-becb-b607099b58e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:38.325264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcf2ab-266a-4671-becb-b607099b58e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:38.326845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcf2ab-266a-4671-becb-b607099b58e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:38.333618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dcf2ab-266a-4671-becb-b607099b58e8 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:38.367493 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 831/1665] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:38.374830 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed5f77-6021-4e32-8416-969be26fe4a3 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:38.412580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed5f77-6021-4e32-8416-969be26fe4a3 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:38.432963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333615d-0e69-46ec-8ea6-030a0a6a5b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:38.434215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333615d-0e69-46ec-8ea6-030a0a6a5b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:38.437667 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333615d-0e69-46ec-8ea6-030a0a6a5b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:38.438359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333615d-0e69-46ec-8ea6-030a0a6a5b25 admin admin] PATH_INFO: `/v3/projects/f378efa87933420687bb35fb71d1e5f8/users/cd5efa6dc7f6441687dd5ac7ad19567f/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:38.444248 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333615d-0e69-46ec-8ea6-030a0a6a5b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f378efa87933420687bb35fb71d1e5f8, user_id=cd5efa6dc7f6441687dd5ac7ad19567f, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:38.447983 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333615d-0e69-46ec-8ea6-030a0a6a5b25 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:38.484340 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 835/1666] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:38 2022] PUT /identity/v3/projects/f378efa87933420687bb35fb71d1e5f8/users/cd5efa6dc7f6441687dd5ac7ad19567f/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:38.509015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d1bcc-6944-4c1c-93b2-bd7caf044384 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:38.509015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d1bcc-6944-4c1c-93b2-bd7caf044384 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:38.509015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d1bcc-6944-4c1c-93b2-bd7caf044384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:38.600291 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-354d1bcc-6944-4c1c-93b2-bd7caf044384 None None] MFA Rules not processed for user `cd5efa6dc7f6441687dd5ac7ad19567f`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:38.617470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-354d1bcc-6944-4c1c-93b2-bd7caf044384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:38.652226 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:38.963079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 832/1667] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:32:38.982910 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333615d-0e69-46ec-8ea6-030a0a6a5b25 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:38.987002 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b333615d-0e69-46ec-8ea6-030a0a6a5b25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:39.264630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333615d-0e69-46ec-8ea6-030a0a6a5b25 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:39.287180 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cd0ea-f834-41c6-9319-f6c5eac486a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:39.290022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cd0ea-f834-41c6-9319-f6c5eac486a6 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:39.290852 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cd0ea-f834-41c6-9319-f6c5eac486a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:39.291614 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cd0ea-f834-41c6-9319-f6c5eac486a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:39.354820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cd0ea-f834-41c6-9319-f6c5eac486a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:39.364089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cd0ea-f834-41c6-9319-f6c5eac486a6 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:39.441414 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 836/1668] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:32:39.932501 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d1bcc-6944-4c1c-93b2-bd7caf044384 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:39.937799 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-354d1bcc-6944-4c1c-93b2-bd7caf044384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:40.299407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d1bcc-6944-4c1c-93b2-bd7caf044384 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:40.309482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0382c1-7b0c-4b49-80dd-89fc4683464f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:40.310947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0382c1-7b0c-4b49-80dd-89fc4683464f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:40.311720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0382c1-7b0c-4b49-80dd-89fc4683464f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:40.312374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0382c1-7b0c-4b49-80dd-89fc4683464f admin admin] PATH_INFO: `/v3/projects/f183fd3d9d744bfb8626e87314fd992e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:40.314858 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0382c1-7b0c-4b49-80dd-89fc4683464f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f183fd3d9d744bfb8626e87314fd992e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:40.320162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0382c1-7b0c-4b49-80dd-89fc4683464f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:40.638496 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-cf0382c1-7b0c-4b49-80dd-89fc4683464f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f183fd3d9d744bfb8626e87314fd992e', 'request_id': 'req-cf0382c1-7b0c-4b49-80dd-89fc4683464f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:40.674729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-cf0382c1-7b0c-4b49-80dd-89fc4683464f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f183fd3d9d744bfb8626e87314fd992e', 'request_id': 'req-cf0382c1-7b0c-4b49-80dd-89fc4683464f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:40.812836 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-cf0382c1-7b0c-4b49-80dd-89fc4683464f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f183fd3d9d744bfb8626e87314fd992e', 'request_id': 'req-cf0382c1-7b0c-4b49-80dd-89fc4683464f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:40.896360 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-cf0382c1-7b0c-4b49-80dd-89fc4683464f admin admin] The token cache is being invalidate because project f183fd3d9d744bfb8626e87314fd992e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:32:40.897349 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-cf0382c1-7b0c-4b49-80dd-89fc4683464f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:40.900884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-cf0382c1-7b0c-4b49-80dd-89fc4683464f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:40.907529 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 833/1669] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:32:39 2022] DELETE /identity/v3/projects/f183fd3d9d744bfb8626e87314fd992e => generated 0 bytes in 973 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:40.914442 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cd0ea-f834-41c6-9319-f6c5eac486a6 service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:40.963977 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cd0ea-f834-41c6-9319-f6c5eac486a6 service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:41.162212 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:41.239999 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4443cb-44ec-4fd7-966a-a78fc7207e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:41.241024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4443cb-44ec-4fd7-966a-a78fc7207e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:41.241505 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4443cb-44ec-4fd7-966a-a78fc7207e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:41.241944 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4443cb-44ec-4fd7-966a-a78fc7207e96 admin admin] PATH_INFO: `/v3/users/dfa1b8332d0841fe9eff9136736b5844` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:41.243856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4443cb-44ec-4fd7-966a-a78fc7207e96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfa1b8332d0841fe9eff9136736b5844)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:41.261456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4443cb-44ec-4fd7-966a-a78fc7207e96 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:41.703811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-cd4443cb-44ec-4fd7-966a-a78fc7207e96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfa1b8332d0841fe9eff9136736b5844'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:41.721682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0382c1-7b0c-4b49-80dd-89fc4683464f admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:41.736675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-cf0382c1-7b0c-4b49-80dd-89fc4683464f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:41.820703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0382c1-7b0c-4b49-80dd-89fc4683464f admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:41.831786 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27331b52-8147-484b-93a1-cdd628fd8f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:41.834183 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27331b52-8147-484b-93a1-cdd628fd8f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:41.836016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27331b52-8147-484b-93a1-cdd628fd8f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:41.836806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27331b52-8147-484b-93a1-cdd628fd8f37 admin admin] PATH_INFO: `/v3/users/3b5aa7791a7f4632af0bc0e90d12173b` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:41.841067 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27331b52-8147-484b-93a1-cdd628fd8f37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b5aa7791a7f4632af0bc0e90d12173b)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:41.845997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27331b52-8147-484b-93a1-cdd628fd8f37 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:41.906316 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-cd4443cb-44ec-4fd7-966a-a78fc7207e96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfa1b8332d0841fe9eff9136736b5844'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:41.924407 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-cd4443cb-44ec-4fd7-966a-a78fc7207e96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfa1b8332d0841fe9eff9136736b5844'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:41.926907 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-cd4443cb-44ec-4fd7-966a-a78fc7207e96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfa1b8332d0841fe9eff9136736b5844'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:41.987202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-cd4443cb-44ec-4fd7-966a-a78fc7207e96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfa1b8332d0841fe9eff9136736b5844'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:42.086835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 837/1670] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:32:40 2022] DELETE /identity/v3/users/dfa1b8332d0841fe9eff9136736b5844 => generated 0 bytes in 1177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:42.093003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b86207-adb3-497a-b03a-c63502aa4b54 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:42.093003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b86207-adb3-497a-b03a-c63502aa4b54 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:42.093003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b86207-adb3-497a-b03a-c63502aa4b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:42.234737 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:42.241676 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-89b86207-adb3-497a-b03a-c63502aa4b54 None None] MFA Rules not processed for user `7f8c7dc2115b44128d800086dee7cfe4`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:42.247201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-89b86207-adb3-497a-b03a-c63502aa4b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:42.333619 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-27331b52-8147-484b-93a1-cdd628fd8f37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b5aa7791a7f4632af0bc0e90d12173b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:42.393203 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-27331b52-8147-484b-93a1-cdd628fd8f37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b5aa7791a7f4632af0bc0e90d12173b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:42.394543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-27331b52-8147-484b-93a1-cdd628fd8f37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b5aa7791a7f4632af0bc0e90d12173b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:42.425946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-27331b52-8147-484b-93a1-cdd628fd8f37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b5aa7791a7f4632af0bc0e90d12173b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:42.434551 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-27331b52-8147-484b-93a1-cdd628fd8f37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b5aa7791a7f4632af0bc0e90d12173b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:42.576376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 834/1671] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:32:41 2022] DELETE /identity/v3/users/3b5aa7791a7f4632af0bc0e90d12173b => generated 0 bytes in 855 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:42.583032 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd734b-1f5e-4f84-b85d-6e83e1ccc663 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:42.586222 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd734b-1f5e-4f84-b85d-6e83e1ccc663 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:42.587587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd734b-1f5e-4f84-b85d-6e83e1ccc663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:42.619071 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 838/1672] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:42 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 533 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:42.699283 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-90fd734b-1f5e-4f84-b85d-6e83e1ccc663 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:42.707254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-90fd734b-1f5e-4f84-b85d-6e83e1ccc663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:43.035571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 835/1673] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:32:43.118641 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b86207-adb3-497a-b03a-c63502aa4b54 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:43.121955 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-89b86207-adb3-497a-b03a-c63502aa4b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:43.317255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:43.361068 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b86207-adb3-497a-b03a-c63502aa4b54 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:43.612979 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c38518-fd4f-40b2-9aa2-2cbf5341b948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:43.615323 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c38518-fd4f-40b2-9aa2-2cbf5341b948 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:43.615930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c38518-fd4f-40b2-9aa2-2cbf5341b948 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:43.616436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c38518-fd4f-40b2-9aa2-2cbf5341b948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:43.641436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c38518-fd4f-40b2-9aa2-2cbf5341b948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:43.646365 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c38518-fd4f-40b2-9aa2-2cbf5341b948 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:43.727567 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 839/1674] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:43.739437 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d58f4d-dc19-43aa-b8b3-62ba738a0032 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:43.739961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d58f4d-dc19-43aa-b8b3-62ba738a0032 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:43.740439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d58f4d-dc19-43aa-b8b3-62ba738a0032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:43.872639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-08d58f4d-dc19-43aa-b8b3-62ba738a0032 None None] MFA Rules not processed for user `a145e63a44db43acafc92bb2a238f463`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:43.889132 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-08d58f4d-dc19-43aa-b8b3-62ba738a0032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:44.217391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 836/1675] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:44.252834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78f0d4-d01a-439d-a7ba-cd204cca698b None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:44.252834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78f0d4-d01a-439d-a7ba-cd204cca698b None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:44.253684 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78f0d4-d01a-439d-a7ba-cd204cca698b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:44.311149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-9c78f0d4-d01a-439d-a7ba-cd204cca698b None None] MFA Rules not processed for user `a145e63a44db43acafc92bb2a238f463`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:44.323816 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9c78f0d4-d01a-439d-a7ba-cd204cca698b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:44.348806 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 840/1676] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:44.372533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d58f4d-dc19-43aa-b8b3-62ba738a0032 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:44.387376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-08d58f4d-dc19-43aa-b8b3-62ba738a0032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:44.517825 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78f0d4-d01a-439d-a7ba-cd204cca698b None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:44.521681 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9c78f0d4-d01a-439d-a7ba-cd204cca698b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:44.569315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:44.613037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78f0d4-d01a-439d-a7ba-cd204cca698b None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:44.625084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc13e53-a304-4ac4-a186-c173be3a2274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:44.626777 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc13e53-a304-4ac4-a186-c173be3a2274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:44.627576 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc13e53-a304-4ac4-a186-c173be3a2274 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:44.628303 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc13e53-a304-4ac4-a186-c173be3a2274 admin admin] PATH_INFO: `/v3/projects/e4531b8689e54cee904a7fb2a4df1d19` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:44.631087 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc13e53-a304-4ac4-a186-c173be3a2274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4531b8689e54cee904a7fb2a4df1d19)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:44.633670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: WARNING py.warnings [None req-8bc13e53-a304-4ac4-a186-c173be3a2274 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 13:32:44.633670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: warnings.warn(msg) Sep 07 13:32:44.633670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: Sep 07 13:32:44.637002 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc13e53-a304-4ac4-a186-c173be3a2274 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:44.715239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d58f4d-dc19-43aa-b8b3-62ba738a0032 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:44.740211 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888f439-2827-4fa0-9508-0fa1c4c3a2d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:44.742880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888f439-2827-4fa0-9508-0fa1c4c3a2d0 service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:44.743926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888f439-2827-4fa0-9508-0fa1c4c3a2d0 service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:44.744686 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888f439-2827-4fa0-9508-0fa1c4c3a2d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:44.772335 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888f439-2827-4fa0-9508-0fa1c4c3a2d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:44.778400 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888f439-2827-4fa0-9508-0fa1c4c3a2d0 service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:44.833535 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 837/1677] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:44 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:32:44.936698 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-8bc13e53-a304-4ac4-a186-c173be3a2274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4531b8689e54cee904a7fb2a4df1d19', 'request_id': 'req-8bc13e53-a304-4ac4-a186-c173be3a2274'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:45.015696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-8bc13e53-a304-4ac4-a186-c173be3a2274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4531b8689e54cee904a7fb2a4df1d19', 'request_id': 'req-8bc13e53-a304-4ac4-a186-c173be3a2274'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:45.230361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-8bc13e53-a304-4ac4-a186-c173be3a2274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4531b8689e54cee904a7fb2a4df1d19', 'request_id': 'req-8bc13e53-a304-4ac4-a186-c173be3a2274'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:45.273219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-8bc13e53-a304-4ac4-a186-c173be3a2274 admin admin] The token cache is being invalidate because project e4531b8689e54cee904a7fb2a4df1d19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:32:45.274180 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-8bc13e53-a304-4ac4-a186-c173be3a2274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:45.276589 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-8bc13e53-a304-4ac4-a186-c173be3a2274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:45.284590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 841/1678] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:32:44 2022] DELETE /identity/v3/projects/e4531b8689e54cee904a7fb2a4df1d19 => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:45.293219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccc096-88bd-449e-87bb-61d6a6a5aa10 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:45.294047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccc096-88bd-449e-87bb-61d6a6a5aa10 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:45.294248 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccc096-88bd-449e-87bb-61d6a6a5aa10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:45.381747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-11ccc096-88bd-449e-87bb-61d6a6a5aa10 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:45.389405 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-11ccc096-88bd-449e-87bb-61d6a6a5aa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:45.549324 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 838/1679] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:45.565610 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a14e05-9933-41c5-a695-0be5ddbd77f9 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:45.566114 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a14e05-9933-41c5-a695-0be5ddbd77f9 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:45.566671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a14e05-9933-41c5-a695-0be5ddbd77f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:45.648912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-72a14e05-9933-41c5-a695-0be5ddbd77f9 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:45.651777 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-72a14e05-9933-41c5-a695-0be5ddbd77f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:45.684660 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 842/1680] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:32:45.691515 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccc096-88bd-449e-87bb-61d6a6a5aa10 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:45.725539 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccc096-88bd-449e-87bb-61d6a6a5aa10 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:45.742571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c6a65-c1d5-42c7-b484-61406f09d598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:45.743714 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c6a65-c1d5-42c7-b484-61406f09d598 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:45.744185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c6a65-c1d5-42c7-b484-61406f09d598 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:45.745273 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c6a65-c1d5-42c7-b484-61406f09d598 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:45.747073 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c6a65-c1d5-42c7-b484-61406f09d598 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:32:45.759898 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c6a65-c1d5-42c7-b484-61406f09d598 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:45.763742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c6a65-c1d5-42c7-b484-61406f09d598 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:45.799610 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 839/1681] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:32:45.806208 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a14e05-9933-41c5-a695-0be5ddbd77f9 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:45.837109 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a14e05-9933-41c5-a695-0be5ddbd77f9 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:45.852221 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99f5c1-a063-4bc6-ac77-12a4afae0996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:45.853298 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99f5c1-a063-4bc6-ac77-12a4afae0996 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:45.853788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99f5c1-a063-4bc6-ac77-12a4afae0996 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:45.854274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99f5c1-a063-4bc6-ac77-12a4afae0996 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:45.855599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99f5c1-a063-4bc6-ac77-12a4afae0996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:45.860245 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99f5c1-a063-4bc6-ac77-12a4afae0996 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:46.137575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 843/1682] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:32:45 2022] POST /identity/v3/projects => generated 387 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:32:46.140617 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c6a65-c1d5-42c7-b484-61406f09d598 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:46.171432 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c6a65-c1d5-42c7-b484-61406f09d598 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:46.360661 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa9b54-3fbf-436f-8391-f3dfe191f1a9 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:46.361263 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa9b54-3fbf-436f-8391-f3dfe191f1a9 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:46.361906 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa9b54-3fbf-436f-8391-f3dfe191f1a9 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:46.364821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 844/1683] 10.222.0.62 () {54 vars in 912 bytes} [Wed Sep 7 13:32:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:32:46.388947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079bd40-8bb3-45ec-84ed-edde170aeeb0 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:46.389856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079bd40-8bb3-45ec-84ed-edde170aeeb0 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:46.390595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079bd40-8bb3-45ec-84ed-edde170aeeb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:46.445867 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3910afd-06de-4206-b807-173c54a915cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:46.447351 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3910afd-06de-4206-b807-173c54a915cd admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:46.449180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3910afd-06de-4206-b807-173c54a915cd admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:46.451252 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3910afd-06de-4206-b807-173c54a915cd admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:46.452860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3910afd-06de-4206-b807-173c54a915cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:46.463739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3910afd-06de-4206-b807-173c54a915cd admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:46.548736 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-7079bd40-8bb3-45ec-84ed-edde170aeeb0 None None] MFA Rules not processed for user `5acdbdb6f23b4f10bcbab0387ea67ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:46.552060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7079bd40-8bb3-45ec-84ed-edde170aeeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:46.639121 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:46.693893 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 845/1684] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 7 13:32:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:32:46.739726 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 840/1685] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:32:46 2022] POST /identity/v3/users => generated 294 bytes in 600 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:46.745840 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079bd40-8bb3-45ec-84ed-edde170aeeb0 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:46.798510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079bd40-8bb3-45ec-84ed-edde170aeeb0 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:46.821267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c597a5-dac5-42e2-b9ef-0564587681d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:46.825736 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c597a5-dac5-42e2-b9ef-0564587681d1 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:46.827308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c597a5-dac5-42e2-b9ef-0564587681d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:46.828342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c597a5-dac5-42e2-b9ef-0564587681d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:46.835439 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c597a5-dac5-42e2-b9ef-0564587681d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:46.841376 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c597a5-dac5-42e2-b9ef-0564587681d1 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:46.899222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 846/1686] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:46.904090 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3910afd-06de-4206-b807-173c54a915cd admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:46.939299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3910afd-06de-4206-b807-173c54a915cd admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:46.960311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efebac-5b7c-479f-b59c-2840b59173e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:46.972185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efebac-5b7c-479f-b59c-2840b59173e2 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:46.972838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efebac-5b7c-479f-b59c-2840b59173e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:46.973359 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efebac-5b7c-479f-b59c-2840b59173e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:46.974725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efebac-5b7c-479f-b59c-2840b59173e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:46.979978 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efebac-5b7c-479f-b59c-2840b59173e2 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:47.052222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c597a5-dac5-42e2-b9ef-0564587681d1 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:47.052948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 841/1687] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:47.093658 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c597a5-dac5-42e2-b9ef-0564587681d1 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:47.123585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4baeea-7e17-4a5b-9c8d-3f4296ea2716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:47.124892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4baeea-7e17-4a5b-9c8d-3f4296ea2716 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:47.125353 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4baeea-7e17-4a5b-9c8d-3f4296ea2716 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:47.125852 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4baeea-7e17-4a5b-9c8d-3f4296ea2716 admin admin] PATH_INFO: `/v3/projects/13c6a875c2ce43a2a9e4c9513a2bf9d6/users/219f85f97f504bf99f1a7fe8b42dc1f3/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:47.182133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4baeea-7e17-4a5b-9c8d-3f4296ea2716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13c6a875c2ce43a2a9e4c9513a2bf9d6, user_id=219f85f97f504bf99f1a7fe8b42dc1f3, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:47.191460 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4baeea-7e17-4a5b-9c8d-3f4296ea2716 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:47.205764 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efebac-5b7c-479f-b59c-2840b59173e2 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:47.208866 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-79efebac-5b7c-479f-b59c-2840b59173e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:47.237214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efebac-5b7c-479f-b59c-2840b59173e2 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:47.253681 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbff81-eb9d-4d6d-91bc-f18efbd45a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:47.255050 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbff81-eb9d-4d6d-91bc-f18efbd45a9b service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:47.255270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbff81-eb9d-4d6d-91bc-f18efbd45a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:47.255694 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbff81-eb9d-4d6d-91bc-f18efbd45a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:47.272398 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbff81-eb9d-4d6d-91bc-f18efbd45a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:47.276480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbff81-eb9d-4d6d-91bc-f18efbd45a9b service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:47.300435 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 842/1688] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:32:47.357554 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 847/1689] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:47 2022] PUT /identity/v3/projects/13c6a875c2ce43a2a9e4c9513a2bf9d6/users/219f85f97f504bf99f1a7fe8b42dc1f3/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:32:47.382651 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5c682-a5c3-4496-8583-a61d1308bd1e None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:47.383306 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5c682-a5c3-4496-8583-a61d1308bd1e None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:47.384156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5c682-a5c3-4496-8583-a61d1308bd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:47.492428 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-b5d5c682-a5c3-4496-8583-a61d1308bd1e None None] MFA Rules not processed for user `219f85f97f504bf99f1a7fe8b42dc1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:47.501838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b5d5c682-a5c3-4496-8583-a61d1308bd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:47.522279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:47.743695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 843/1690] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:47.745779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4baeea-7e17-4a5b-9c8d-3f4296ea2716 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:48.157650 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4baeea-7e17-4a5b-9c8d-3f4296ea2716 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:48.185807 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce03c70-5d9c-49a6-885c-995a5aa40ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:48.189501 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce03c70-5d9c-49a6-885c-995a5aa40ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:48.191493 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce03c70-5d9c-49a6-885c-995a5aa40ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:48.192107 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce03c70-5d9c-49a6-885c-995a5aa40ee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:48.194821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce03c70-5d9c-49a6-885c-995a5aa40ee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:48.210910 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce03c70-5d9c-49a6-885c-995a5aa40ee2 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:48.422729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 848/1691] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:32:47 2022] POST /identity/v3/projects => generated 387 bytes in 673 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:48.424901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5c682-a5c3-4496-8583-a61d1308bd1e None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:48.471128 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5c682-a5c3-4496-8583-a61d1308bd1e None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:48.754516 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c9403-22a4-44c0-a60e-488d039ea4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:48.755681 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c9403-22a4-44c0-a60e-488d039ea4f2 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:48.756114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c9403-22a4-44c0-a60e-488d039ea4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:48.756581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c9403-22a4-44c0-a60e-488d039ea4f2 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:48.757888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06c9403-22a4-44c0-a60e-488d039ea4f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:48.762435 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06c9403-22a4-44c0-a60e-488d039ea4f2 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:48.950189 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 844/1692] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:32:48 2022] POST /identity/v3/users => generated 300 bytes in 521 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:32:48.952234 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce03c70-5d9c-49a6-885c-995a5aa40ee2 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:48.988848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce03c70-5d9c-49a6-885c-995a5aa40ee2 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:49.013832 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd8448-04e9-4de9-8df4-7c82d3f8a347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:49.014946 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd8448-04e9-4de9-8df4-7c82d3f8a347 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:49.015639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd8448-04e9-4de9-8df4-7c82d3f8a347 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:49.016131 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd8448-04e9-4de9-8df4-7c82d3f8a347 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:49.018178 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd8448-04e9-4de9-8df4-7c82d3f8a347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:49.024470 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd8448-04e9-4de9-8df4-7c82d3f8a347 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:49.077924 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 849/1693] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:32:49.084217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c9403-22a4-44c0-a60e-488d039ea4f2 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:49.123827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c9403-22a4-44c0-a60e-488d039ea4f2 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:49.142595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd607b34-4134-4352-987e-01512a943d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:49.144109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd607b34-4134-4352-987e-01512a943d79 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:49.149656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd607b34-4134-4352-987e-01512a943d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:49.150311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd607b34-4134-4352-987e-01512a943d79 admin admin] PATH_INFO: `/v3/projects/c510ebda3c4144b29607b7fe17f271fc/users/0b13803ed86d4e23b39c2cb8bf299298/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:49.197166 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd607b34-4134-4352-987e-01512a943d79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c510ebda3c4144b29607b7fe17f271fc, user_id=0b13803ed86d4e23b39c2cb8bf299298, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:49.204861 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd607b34-4134-4352-987e-01512a943d79 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:49.410811 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 845/1694] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:49 2022] PUT /identity/v3/projects/c510ebda3c4144b29607b7fe17f271fc/users/0b13803ed86d4e23b39c2cb8bf299298/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:49.419042 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd8448-04e9-4de9-8df4-7c82d3f8a347 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:49.474918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd8448-04e9-4de9-8df4-7c82d3f8a347 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:49.743726 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd607b34-4134-4352-987e-01512a943d79 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:49.747077 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-fd607b34-4134-4352-987e-01512a943d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:49.787239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5e6a3-e377-429b-b339-5162e63206ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:49.788519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5e6a3-e377-429b-b339-5162e63206ba admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:49.788992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5e6a3-e377-429b-b339-5162e63206ba admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:49.789471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5e6a3-e377-429b-b339-5162e63206ba admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:49.791725 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5e6a3-e377-429b-b339-5162e63206ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:49.796166 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5e6a3-e377-429b-b339-5162e63206ba admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:49.847177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 850/1695] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:32:49.865754 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5e6a3-e377-429b-b339-5162e63206ba admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:49.933073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5e6a3-e377-429b-b339-5162e63206ba admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:49.968351 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953a81b-db59-4d62-a78a-29ef730d7c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:49.975438 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953a81b-db59-4d62-a78a-29ef730d7c59 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:49.978636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953a81b-db59-4d62-a78a-29ef730d7c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:49.979193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953a81b-db59-4d62-a78a-29ef730d7c59 admin admin] PATH_INFO: `/v3/projects/c510ebda3c4144b29607b7fe17f271fc/users/0b13803ed86d4e23b39c2cb8bf299298/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:49.995301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953a81b-db59-4d62-a78a-29ef730d7c59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c510ebda3c4144b29607b7fe17f271fc, user_id=0b13803ed86d4e23b39c2cb8bf299298, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:50.011377 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953a81b-db59-4d62-a78a-29ef730d7c59 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:50.122870 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 851/1696] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:32:49 2022] PUT /identity/v3/projects/c510ebda3c4144b29607b7fe17f271fc/users/0b13803ed86d4e23b39c2cb8bf299298/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:50.130636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd607b34-4134-4352-987e-01512a943d79 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:50.132332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953a81b-db59-4d62-a78a-29ef730d7c59 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:50.143463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-6953a81b-db59-4d62-a78a-29ef730d7c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:50.169051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:50.198072 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:50.580502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953a81b-db59-4d62-a78a-29ef730d7c59 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:50.600222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705d9ca-f437-4176-a2df-a5d5e8d0c8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:50.608821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705d9ca-f437-4176-a2df-a5d5e8d0c8d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:50.609290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705d9ca-f437-4176-a2df-a5d5e8d0c8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:50.609799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705d9ca-f437-4176-a2df-a5d5e8d0c8d9 admin admin] PATH_INFO: `/v3/projects/74213b06d3dd4e5998a78692c52e1970` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:50.616294 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705d9ca-f437-4176-a2df-a5d5e8d0c8d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74213b06d3dd4e5998a78692c52e1970)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:50.620497 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0a820-018d-4d98-b439-7aef2820f95e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bee90bf951f04c2e8e19ceeacb8cfcb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:50.626147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705d9ca-f437-4176-a2df-a5d5e8d0c8d9 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:50.626679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0a820-018d-4d98-b439-7aef2820f95e service placement] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:50.633053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0a820-018d-4d98-b439-7aef2820f95e service placement] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:50.636153 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0a820-018d-4d98-b439-7aef2820f95e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:50.646963 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-93b0a820-018d-4d98-b439-7aef2820f95e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:50.710501 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0a820-018d-4d98-b439-7aef2820f95e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:50.716076 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b0a820-018d-4d98-b439-7aef2820f95e service placement] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:50.797896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 846/1697] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 1055 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:32:50.804995 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f1930-0103-490f-b72c-8d6e84007d34 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:50.805996 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f1930-0103-490f-b72c-8d6e84007d34 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:50.806737 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f1930-0103-490f-b72c-8d6e84007d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:50.942755 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e705d9ca-f437-4176-a2df-a5d5e8d0c8d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74213b06d3dd4e5998a78692c52e1970', 'request_id': 'req-e705d9ca-f437-4176-a2df-a5d5e8d0c8d9'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:50.965330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-560f1930-0103-490f-b72c-8d6e84007d34 None None] MFA Rules not processed for user `0b13803ed86d4e23b39c2cb8bf299298`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:50.973421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-560f1930-0103-490f-b72c-8d6e84007d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:50.984139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e705d9ca-f437-4176-a2df-a5d5e8d0c8d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74213b06d3dd4e5998a78692c52e1970', 'request_id': 'req-e705d9ca-f437-4176-a2df-a5d5e8d0c8d9'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:51.121618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e705d9ca-f437-4176-a2df-a5d5e8d0c8d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74213b06d3dd4e5998a78692c52e1970', 'request_id': 'req-e705d9ca-f437-4176-a2df-a5d5e8d0c8d9'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:51.177895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e705d9ca-f437-4176-a2df-a5d5e8d0c8d9 admin admin] The token cache is being invalidate because project 74213b06d3dd4e5998a78692c52e1970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:32:51.179679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e705d9ca-f437-4176-a2df-a5d5e8d0c8d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:51.183954 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e705d9ca-f437-4176-a2df-a5d5e8d0c8d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:51.195834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 852/1698] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:32:50 2022] DELETE /identity/v3/projects/74213b06d3dd4e5998a78692c52e1970 => generated 0 bytes in 1060 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:51.201387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725a41e-a3e0-4200-b8c3-cc75dbc7ae13 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:51.203890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725a41e-a3e0-4200-b8c3-cc75dbc7ae13 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:51.205942 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725a41e-a3e0-4200-b8c3-cc75dbc7ae13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:51.289992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-1725a41e-a3e0-4200-b8c3-cc75dbc7ae13 None None] MFA Rules not processed for user `a145e63a44db43acafc92bb2a238f463`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:51.296359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1725a41e-a3e0-4200-b8c3-cc75dbc7ae13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:51.472772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 853/1699] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:51.474181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725a41e-a3e0-4200-b8c3-cc75dbc7ae13 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:51.492836 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1725a41e-a3e0-4200-b8c3-cc75dbc7ae13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:51.629907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 847/1700] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 831 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:51.636049 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f1930-0103-490f-b72c-8d6e84007d34 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:51.647216 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-560f1930-0103-490f-b72c-8d6e84007d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:51.843668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:51.944774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725a41e-a3e0-4200-b8c3-cc75dbc7ae13 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:51.970686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14ddd1-6991-4194-b1b4-4d624881f99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:51.972206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14ddd1-6991-4194-b1b4-4d624881f99e admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:51.974286 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f1930-0103-490f-b72c-8d6e84007d34 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:51.976501 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14ddd1-6991-4194-b1b4-4d624881f99e admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:51.977551 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14ddd1-6991-4194-b1b4-4d624881f99e admin admin] PATH_INFO: `/v3/users/cd5efa6dc7f6441687dd5ac7ad19567f` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:51.980254 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14ddd1-6991-4194-b1b4-4d624881f99e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd5efa6dc7f6441687dd5ac7ad19567f)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:51.984347 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b76336-b476-4672-b97c-f0f971fe3de5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:51.985467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b76336-b476-4672-b97c-f0f971fe3de5 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:51.985982 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b76336-b476-4672-b97c-f0f971fe3de5 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:51.986491 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b76336-b476-4672-b97c-f0f971fe3de5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:52.008408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14ddd1-6991-4194-b1b4-4d624881f99e admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:52.039065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b76336-b476-4672-b97c-f0f971fe3de5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:52.042093 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b76336-b476-4672-b97c-f0f971fe3de5 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:52.102507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 848/1701] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:51 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:52.105119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718e9f6-d0b4-40b5-9849-00164ea06e9f None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:52.111970 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718e9f6-d0b4-40b5-9849-00164ea06e9f None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:52.112474 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718e9f6-d0b4-40b5-9849-00164ea06e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:52.248851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-0718e9f6-d0b4-40b5-9849-00164ea06e9f None None] MFA Rules not processed for user `7e7db29f64934026abbe96d5cb5baf25`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:52.254209 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-0718e9f6-d0b4-40b5-9849-00164ea06e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:52.319143 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:52.536098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-8b14ddd1-6991-4194-b1b4-4d624881f99e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd5efa6dc7f6441687dd5ac7ad19567f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:52.629940 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 849/1702] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 526 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:52.634050 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718e9f6-d0b4-40b5-9849-00164ea06e9f None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:52.635923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-0718e9f6-d0b4-40b5-9849-00164ea06e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:52.653746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-8b14ddd1-6991-4194-b1b4-4d624881f99e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd5efa6dc7f6441687dd5ac7ad19567f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:52.660860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-8b14ddd1-6991-4194-b1b4-4d624881f99e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd5efa6dc7f6441687dd5ac7ad19567f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:52.660860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-8b14ddd1-6991-4194-b1b4-4d624881f99e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd5efa6dc7f6441687dd5ac7ad19567f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:52.693101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-8b14ddd1-6991-4194-b1b4-4d624881f99e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd5efa6dc7f6441687dd5ac7ad19567f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:52.724753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718e9f6-d0b4-40b5-9849-00164ea06e9f None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:52.750700 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819fd9a-d670-4523-a434-03a652129814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:52.751951 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819fd9a-d670-4523-a434-03a652129814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:52.752444 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819fd9a-d670-4523-a434-03a652129814 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:52.752946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819fd9a-d670-4523-a434-03a652129814 admin admin] PATH_INFO: `/v3/users/219f85f97f504bf99f1a7fe8b42dc1f3` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:52.755320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819fd9a-d670-4523-a434-03a652129814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=219f85f97f504bf99f1a7fe8b42dc1f3)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:52.755920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 854/1703] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:32:51 2022] DELETE /identity/v3/users/cd5efa6dc7f6441687dd5ac7ad19567f => generated 0 bytes in 1283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:52.760238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14ddd1-6991-4194-b1b4-4d624881f99e admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:52.764916 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8b14ddd1-6991-4194-b1b4-4d624881f99e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:52.765233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819fd9a-d670-4523-a434-03a652129814 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:53.109171 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-7819fd9a-d670-4523-a434-03a652129814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '219f85f97f504bf99f1a7fe8b42dc1f3'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:53.112869 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14ddd1-6991-4194-b1b4-4d624881f99e admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:53.122813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af1c2a-a0c8-406a-a379-16bde23756c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:53.123904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af1c2a-a0c8-406a-a379-16bde23756c5 service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:53.125331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af1c2a-a0c8-406a-a379-16bde23756c5 service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:53.127066 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af1c2a-a0c8-406a-a379-16bde23756c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:53.131866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f1af1c2a-a0c8-406a-a379-16bde23756c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:53.173192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-7819fd9a-d670-4523-a434-03a652129814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '219f85f97f504bf99f1a7fe8b42dc1f3'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:53.180558 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-7819fd9a-d670-4523-a434-03a652129814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '219f85f97f504bf99f1a7fe8b42dc1f3'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:53.202654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-7819fd9a-d670-4523-a434-03a652129814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '219f85f97f504bf99f1a7fe8b42dc1f3'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:53.204189 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-7819fd9a-d670-4523-a434-03a652129814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '219f85f97f504bf99f1a7fe8b42dc1f3'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:53.395440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 850/1704] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:32:52 2022] DELETE /identity/v3/users/219f85f97f504bf99f1a7fe8b42dc1f3 => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:53.398258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47907cb-9702-41b2-a76a-b05bab3b7d52 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:53.405499 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47907cb-9702-41b2-a76a-b05bab3b7d52 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:53.406383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47907cb-9702-41b2-a76a-b05bab3b7d52 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:53.406638 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af1c2a-a0c8-406a-a379-16bde23756c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:53.410033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af1c2a-a0c8-406a-a379-16bde23756c5 service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:53.411043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 851/1705] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 7 13:32:53 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:32:53.413656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403de41-e1d3-4936-b2cb-c91789af7010 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:53.431248 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403de41-e1d3-4936-b2cb-c91789af7010 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:53.432263 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403de41-e1d3-4936-b2cb-c91789af7010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:53.491830 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:53.499564 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-6403de41-e1d3-4936-b2cb-c91789af7010 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:53.505608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6403de41-e1d3-4936-b2cb-c91789af7010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:53.518617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 855/1706] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 762 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:53.529140 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465da769-4ed8-4e1c-91c6-4ed3e0358237 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:53.530815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465da769-4ed8-4e1c-91c6-4ed3e0358237 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:53.531647 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465da769-4ed8-4e1c-91c6-4ed3e0358237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:53.639238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-465da769-4ed8-4e1c-91c6-4ed3e0358237 None None] MFA Rules not processed for user `bee90bf951f04c2e8e19ceeacb8cfcb2`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:53.642538 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-465da769-4ed8-4e1c-91c6-4ed3e0358237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:53.843455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 852/1707] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:53.900927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403de41-e1d3-4936-b2cb-c91789af7010 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:53.904650 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6403de41-e1d3-4936-b2cb-c91789af7010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:53.957962 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 856/1708] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 7 13:32:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:54.050930 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:54.093264 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7a2ce-0cb2-4a57-9e72-f10c6f6c5692 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:54.093828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7a2ce-0cb2-4a57-9e72-f10c6f6c5692 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:54.094285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7a2ce-0cb2-4a57-9e72-f10c6f6c5692 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:54.097026 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 857/1709] 10.222.0.62 () {54 vars in 961 bytes} [Wed Sep 7 13:32:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:32:54.117398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf51bfd-9bad-487f-b302-9e62afcd6377 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:54.118074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf51bfd-9bad-487f-b302-9e62afcd6377 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:54.118409 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf51bfd-9bad-487f-b302-9e62afcd6377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:54.136475 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403de41-e1d3-4936-b2cb-c91789af7010 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:54.144468 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d227a1-99eb-46d7-b212-e22ef59cf417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:54.147769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d227a1-99eb-46d7-b212-e22ef59cf417 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:54.148234 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d227a1-99eb-46d7-b212-e22ef59cf417 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:54.148676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d227a1-99eb-46d7-b212-e22ef59cf417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:54.175412 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d227a1-99eb-46d7-b212-e22ef59cf417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:54.190497 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-3cf51bfd-9bad-487f-b302-9e62afcd6377 None None] MFA Rules not processed for user `bee90bf951f04c2e8e19ceeacb8cfcb2`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:54.190497 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-3cf51bfd-9bad-487f-b302-9e62afcd6377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:54.190794 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d227a1-99eb-46d7-b212-e22ef59cf417 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:54.201385 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 858/1710] 10.222.0.62 () {58 vars in 1058 bytes} [Wed Sep 7 13:32:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:32:54.217041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11e43f-999f-430c-95f2-f26b13988ec6 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:54.217517 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11e43f-999f-430c-95f2-f26b13988ec6 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:54.218358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11e43f-999f-430c-95f2-f26b13988ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:54.226263 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 853/1711] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:54.227961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d227a1-99eb-46d7-b212-e22ef59cf417 service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:54.257250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d227a1-99eb-46d7-b212-e22ef59cf417 service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:54.267146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f8429-9c54-4065-a2f8-2df8cbf9c94c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bee90bf951f04c2e8e19ceeacb8cfcb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:54.268498 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f8429-9c54-4065-a2f8-2df8cbf9c94c service placement] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:54.268857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f8429-9c54-4065-a2f8-2df8cbf9c94c service placement] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:54.269991 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f8429-9c54-4065-a2f8-2df8cbf9c94c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:54.309934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-9a11e43f-999f-430c-95f2-f26b13988ec6 None None] MFA Rules not processed for user `80341b19855e496b96b2948b916db613`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:54.318411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f8429-9c54-4065-a2f8-2df8cbf9c94c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:54.318780 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9a11e43f-999f-430c-95f2-f26b13988ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:54.329727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f8429-9c54-4065-a2f8-2df8cbf9c94c service placement] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:54.405237 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 854/1712] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:54.582963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 859/1713] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:54.602593 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac13058-ed61-4e96-bd38-14d5505fd129 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:54.603213 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac13058-ed61-4e96-bd38-14d5505fd129 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:54.603755 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac13058-ed61-4e96-bd38-14d5505fd129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:54.680465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-fac13058-ed61-4e96-bd38-14d5505fd129 None None] MFA Rules not processed for user `80341b19855e496b96b2948b916db613`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:54.681979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-fac13058-ed61-4e96-bd38-14d5505fd129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:54.718039 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 855/1714] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:54.742319 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a11e43f-999f-430c-95f2-f26b13988ec6 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:54.745768 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9a11e43f-999f-430c-95f2-f26b13988ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:55.094854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a11e43f-999f-430c-95f2-f26b13988ec6 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:55.137882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7517ffec-5c97-42e0-a759-83a9f78fe4df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:55.141300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517ffec-5c97-42e0-a759-83a9f78fe4df service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:55.147601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517ffec-5c97-42e0-a759-83a9f78fe4df service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:55.150507 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517ffec-5c97-42e0-a759-83a9f78fe4df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:55.222034 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7517ffec-5c97-42e0-a759-83a9f78fe4df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:55.247889 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7517ffec-5c97-42e0-a759-83a9f78fe4df service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:55.281574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 860/1715] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:56.538237 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac13058-ed61-4e96-bd38-14d5505fd129 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:56.541097 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-fac13058-ed61-4e96-bd38-14d5505fd129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:56.602281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac13058-ed61-4e96-bd38-14d5505fd129 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:56.628814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848307e-41e1-41ed-9f5e-a8bd1e76be13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:56.632806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848307e-41e1-41ed-9f5e-a8bd1e76be13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:56.635469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848307e-41e1-41ed-9f5e-a8bd1e76be13 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:56.639835 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848307e-41e1-41ed-9f5e-a8bd1e76be13 admin admin] PATH_INFO: `/v3/projects/f378efa87933420687bb35fb71d1e5f8` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:56.644269 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848307e-41e1-41ed-9f5e-a8bd1e76be13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f378efa87933420687bb35fb71d1e5f8)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:56.666467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848307e-41e1-41ed-9f5e-a8bd1e76be13 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:56.831453 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106aafd-38a1-4345-8613-e030b558c1cc None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:56.832344 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106aafd-38a1-4345-8613-e030b558c1cc None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:56.832344 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106aafd-38a1-4345-8613-e030b558c1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:56.868811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:57.002533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-f106aafd-38a1-4345-8613-e030b558c1cc None None] MFA Rules not processed for user `c5e931aa22344990a0f032159ebc20ad`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:57.012800 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f106aafd-38a1-4345-8613-e030b558c1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:57.026255 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-8848307e-41e1-41ed-9f5e-a8bd1e76be13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f378efa87933420687bb35fb71d1e5f8', 'request_id': 'req-8848307e-41e1-41ed-9f5e-a8bd1e76be13'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:57.080633 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-8848307e-41e1-41ed-9f5e-a8bd1e76be13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f378efa87933420687bb35fb71d1e5f8', 'request_id': 'req-8848307e-41e1-41ed-9f5e-a8bd1e76be13'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:57.206260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-8848307e-41e1-41ed-9f5e-a8bd1e76be13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f378efa87933420687bb35fb71d1e5f8', 'request_id': 'req-8848307e-41e1-41ed-9f5e-a8bd1e76be13'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:57.268708 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-8848307e-41e1-41ed-9f5e-a8bd1e76be13 admin admin] The token cache is being invalidate because project f378efa87933420687bb35fb71d1e5f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:32:57.269235 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-8848307e-41e1-41ed-9f5e-a8bd1e76be13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:57.273987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-8848307e-41e1-41ed-9f5e-a8bd1e76be13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:57.284439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 856/1716] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:32:56 2022] DELETE /identity/v3/projects/f378efa87933420687bb35fb71d1e5f8 => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:57.286250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848307e-41e1-41ed-9f5e-a8bd1e76be13 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:57.296243 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-8848307e-41e1-41ed-9f5e-a8bd1e76be13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:57.481861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 861/1717] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 653 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:32:57.486076 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9c769-ed81-43ef-985b-fd7a73b3bae1 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:57.486582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9c769-ed81-43ef-985b-fd7a73b3bae1 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:57.487093 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9c769-ed81-43ef-985b-fd7a73b3bae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:57.636665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-80b9c769-ed81-43ef-985b-fd7a73b3bae1 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:57.639730 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-80b9c769-ed81-43ef-985b-fd7a73b3bae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:57.772370 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848307e-41e1-41ed-9f5e-a8bd1e76be13 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:57.801979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b654dd-9d31-41a9-bbf5-63d71c4f1511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:57.803121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b654dd-9d31-41a9-bbf5-63d71c4f1511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:57.803686 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b654dd-9d31-41a9-bbf5-63d71c4f1511 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:57.804123 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b654dd-9d31-41a9-bbf5-63d71c4f1511 admin admin] PATH_INFO: `/v3/projects/13c6a875c2ce43a2a9e4c9513a2bf9d6` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:57.815822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b654dd-9d31-41a9-bbf5-63d71c4f1511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13c6a875c2ce43a2a9e4c9513a2bf9d6)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:57.819956 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b654dd-9d31-41a9-bbf5-63d71c4f1511 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:58.048414 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 862/1718] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 565 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:58.050546 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9c769-ed81-43ef-985b-fd7a73b3bae1 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:58.059159 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-80b9c769-ed81-43ef-985b-fd7a73b3bae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:58.162488 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:58.180512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-77b654dd-9d31-41a9-bbf5-63d71c4f1511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13c6a875c2ce43a2a9e4c9513a2bf9d6', 'request_id': 'req-77b654dd-9d31-41a9-bbf5-63d71c4f1511'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:58.233806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-77b654dd-9d31-41a9-bbf5-63d71c4f1511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13c6a875c2ce43a2a9e4c9513a2bf9d6', 'request_id': 'req-77b654dd-9d31-41a9-bbf5-63d71c4f1511'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:58.371723 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-77b654dd-9d31-41a9-bbf5-63d71c4f1511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13c6a875c2ce43a2a9e4c9513a2bf9d6', 'request_id': 'req-77b654dd-9d31-41a9-bbf5-63d71c4f1511'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:58.424231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-77b654dd-9d31-41a9-bbf5-63d71c4f1511 admin admin] The token cache is being invalidate because project 13c6a875c2ce43a2a9e4c9513a2bf9d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:32:58.675619 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-77b654dd-9d31-41a9-bbf5-63d71c4f1511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:58.675619 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-77b654dd-9d31-41a9-bbf5-63d71c4f1511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:58.675619 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 857/1719] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:32:57 2022] DELETE /identity/v3/projects/13c6a875c2ce43a2a9e4c9513a2bf9d6 => generated 0 bytes in 1152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:58.675619 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37424aa4-59e9-45f9-aa93-b16747349f88 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:58.675619 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37424aa4-59e9-45f9-aa93-b16747349f88 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:58.675619 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37424aa4-59e9-45f9-aa93-b16747349f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:58.677607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9c769-ed81-43ef-985b-fd7a73b3bae1 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:58.677816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-37424aa4-59e9-45f9-aa93-b16747349f88 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:58.677816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-37424aa4-59e9-45f9-aa93-b16747349f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:58.678067 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:32:58.787868 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde79277-323f-47aa-bbc9-2a8a5591108e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:58.789984 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde79277-323f-47aa-bbc9-2a8a5591108e service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:58.790661 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde79277-323f-47aa-bbc9-2a8a5591108e service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:58.791856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde79277-323f-47aa-bbc9-2a8a5591108e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:58.798636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-cde79277-323f-47aa-bbc9-2a8a5591108e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:58.921763 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 858/1720] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:58.926075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cf575-8172-40e2-b234-0a0b6b99a43c None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:58.926454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cf575-8172-40e2-b234-0a0b6b99a43c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:58.926932 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cf575-8172-40e2-b234-0a0b6b99a43c None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:58.936954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 859/1721] 10.222.0.62 () {54 vars in 912 bytes} [Wed Sep 7 13:32:58 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:32:58.942404 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cf575-8172-40e2-b234-0a0b6b99a43c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:58.947927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-072cf575-8172-40e2-b234-0a0b6b99a43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:59.033498 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cf575-8172-40e2-b234-0a0b6b99a43c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:59.071783 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c20621-70f8-4030-a035-1073367eeafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:59.077636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c20621-70f8-4030-a035-1073367eeafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:59.082409 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c20621-70f8-4030-a035-1073367eeafd admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:59.084306 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c20621-70f8-4030-a035-1073367eeafd admin admin] PATH_INFO: `/v3/users/0b13803ed86d4e23b39c2cb8bf299298` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:59.093009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c20621-70f8-4030-a035-1073367eeafd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b13803ed86d4e23b39c2cb8bf299298)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:59.106130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c20621-70f8-4030-a035-1073367eeafd admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:59.138407 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde79277-323f-47aa-bbc9-2a8a5591108e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:59.141062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde79277-323f-47aa-bbc9-2a8a5591108e service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:59.164349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 863/1722] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 1115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:32:59.167112 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde79277-323f-47aa-bbc9-2a8a5591108e service nova] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:59.221446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde79277-323f-47aa-bbc9-2a8a5591108e service nova] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:59.239021 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372b5c3-cff0-4724-9343-17ebe7bcd0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:59.239021 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372b5c3-cff0-4724-9343-17ebe7bcd0c7 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:59.239804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372b5c3-cff0-4724-9343-17ebe7bcd0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:59.239804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372b5c3-cff0-4724-9343-17ebe7bcd0c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:59.241496 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372b5c3-cff0-4724-9343-17ebe7bcd0c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:32:59.241496 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372b5c3-cff0-4724-9343-17ebe7bcd0c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:59.251600 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372b5c3-cff0-4724-9343-17ebe7bcd0c7 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:59.339218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 864/1723] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:32:59.346118 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aab74-8e72-435e-8f5e-33a39e0becc7 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:59.347766 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aab74-8e72-435e-8f5e-33a39e0becc7 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:59.351170 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aab74-8e72-435e-8f5e-33a39e0becc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:59.433529 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-795aab74-8e72-435e-8f5e-33a39e0becc7 None None] MFA Rules not processed for user `5acdbdb6f23b4f10bcbab0387ea67ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:32:59.440644 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-795aab74-8e72-435e-8f5e-33a39e0becc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:59.567521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a7c20621-70f8-4030-a035-1073367eeafd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b13803ed86d4e23b39c2cb8bf299298'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:59.626609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a7c20621-70f8-4030-a035-1073367eeafd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b13803ed86d4e23b39c2cb8bf299298'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:59.630758 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a7c20621-70f8-4030-a035-1073367eeafd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b13803ed86d4e23b39c2cb8bf299298'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:59.693391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a7c20621-70f8-4030-a035-1073367eeafd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b13803ed86d4e23b39c2cb8bf299298'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:59.699627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a7c20621-70f8-4030-a035-1073367eeafd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b13803ed86d4e23b39c2cb8bf299298'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:32:59.699894 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 865/1724] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 7 13:32:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:32:59.700197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aab74-8e72-435e-8f5e-33a39e0becc7 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:59.705841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-795aab74-8e72-435e-8f5e-33a39e0becc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:32:59.768666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aab74-8e72-435e-8f5e-33a39e0becc7 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:32:59.795856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324aa24d-9178-4048-ae3b-1143987aef1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:32:59.795856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324aa24d-9178-4048-ae3b-1143987aef1f admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:32:59.803926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324aa24d-9178-4048-ae3b-1143987aef1f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:32:59.803926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324aa24d-9178-4048-ae3b-1143987aef1f admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:32:59.803926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324aa24d-9178-4048-ae3b-1143987aef1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:32:59.811575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324aa24d-9178-4048-ae3b-1143987aef1f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:32:59.875078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 860/1725] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:32:58 2022] DELETE /identity/v3/users/0b13803ed86d4e23b39c2cb8bf299298 => generated 0 bytes in 934 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:32:59.876171 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c20621-70f8-4030-a035-1073367eeafd admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:32:59.883220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a7c20621-70f8-4030-a035-1073367eeafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:00.015809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 866/1726] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:32:59 2022] POST /identity/v3/projects => generated 371 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:33:00.019275 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:00.025540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324aa24d-9178-4048-ae3b-1143987aef1f admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:00.061101 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c20621-70f8-4030-a035-1073367eeafd admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:00.340927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:00.375358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd148c-55be-4be7-a6d4-d6ec52d7cf15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:00.376194 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd148c-55be-4be7-a6d4-d6ec52d7cf15 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:00.376672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd148c-55be-4be7-a6d4-d6ec52d7cf15 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:00.377138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd148c-55be-4be7-a6d4-d6ec52d7cf15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:00.424457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324aa24d-9178-4048-ae3b-1143987aef1f admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:00.424918 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd148c-55be-4be7-a6d4-d6ec52d7cf15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:00.427852 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd148c-55be-4be7-a6d4-d6ec52d7cf15 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:00.449894 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b8b29-bad2-4c8d-8e6e-d5ba5c41976a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:00.451509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b8b29-bad2-4c8d-8e6e-d5ba5c41976a admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:00.452651 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b8b29-bad2-4c8d-8e6e-d5ba5c41976a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:00.453459 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b8b29-bad2-4c8d-8e6e-d5ba5c41976a admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:00.454968 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b8b29-bad2-4c8d-8e6e-d5ba5c41976a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:00.459626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b8b29-bad2-4c8d-8e6e-d5ba5c41976a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:00.472664 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 861/1727] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:32:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:33:00.485732 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865e40f-c0c9-476d-bea6-1939161a689a None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:00.486301 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865e40f-c0c9-476d-bea6-1939161a689a None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:00.487027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865e40f-c0c9-476d-bea6-1939161a689a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:00.687400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 867/1728] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:33:00 2022] POST /identity/v3/users => generated 286 bytes in 666 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:33:00.689242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b8b29-bad2-4c8d-8e6e-d5ba5c41976a admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:00.712475 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-1865e40f-c0c9-476d-bea6-1939161a689a None None] MFA Rules not processed for user `8121ba4cf2cf44d9a5a55975ea85928c`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:00.716058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-1865e40f-c0c9-476d-bea6-1939161a689a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:00.743452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b8b29-bad2-4c8d-8e6e-d5ba5c41976a admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:00.784358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fecc7-8967-448f-b975-e8f7001fcc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:00.787602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fecc7-8967-448f-b975-e8f7001fcc2b admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:00.788151 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fecc7-8967-448f-b975-e8f7001fcc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:00.788642 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fecc7-8967-448f-b975-e8f7001fcc2b admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:00.789949 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132fecc7-8967-448f-b975-e8f7001fcc2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:00.793693 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132fecc7-8967-448f-b975-e8f7001fcc2b admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:00.872270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 868/1729] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:33:00.875838 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fecc7-8967-448f-b975-e8f7001fcc2b admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:00.949639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fecc7-8967-448f-b975-e8f7001fcc2b admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:00.949639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f815af-5071-4e7f-a3ce-f5ee64150f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:00.949639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f815af-5071-4e7f-a3ce-f5ee64150f16 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:00.952181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f815af-5071-4e7f-a3ce-f5ee64150f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:00.956968 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f815af-5071-4e7f-a3ce-f5ee64150f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:00.958664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f815af-5071-4e7f-a3ce-f5ee64150f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:00.965902 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f815af-5071-4e7f-a3ce-f5ee64150f16 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:01.026075 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 869/1730] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:33:01.027496 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 862/1731] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 7 13:33:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 552 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:01.029435 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865e40f-c0c9-476d-bea6-1939161a689a None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:01.089505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865e40f-c0c9-476d-bea6-1939161a689a None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:01.102606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831aded6-473d-4bb3-ac28-ec0a96dc9be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:01.104503 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831aded6-473d-4bb3-ac28-ec0a96dc9be3 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:01.105801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831aded6-473d-4bb3-ac28-ec0a96dc9be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:01.106547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831aded6-473d-4bb3-ac28-ec0a96dc9be3 admin admin] PATH_INFO: `/v3/projects/c962b174c7004e5eac678e8e96e81560/users/3a9a1057fff246d7b41234c073cdb914/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:01.187995 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831aded6-473d-4bb3-ac28-ec0a96dc9be3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c962b174c7004e5eac678e8e96e81560, user_id=3a9a1057fff246d7b41234c073cdb914, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:01.202369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831aded6-473d-4bb3-ac28-ec0a96dc9be3 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:01.501406 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 863/1732] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:33:01 2022] PUT /identity/v3/projects/c962b174c7004e5eac678e8e96e81560/users/3a9a1057fff246d7b41234c073cdb914/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:01.509984 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92f819-5e8f-4b01-a655-7e36d2d5b28b None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:01.510819 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92f819-5e8f-4b01-a655-7e36d2d5b28b None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:01.511206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92f819-5e8f-4b01-a655-7e36d2d5b28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:01.629829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-ac92f819-5e8f-4b01-a655-7e36d2d5b28b None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:01.634461 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ac92f819-5e8f-4b01-a655-7e36d2d5b28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:01.814648 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831aded6-473d-4bb3-ac28-ec0a96dc9be3 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:01.817048 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-831aded6-473d-4bb3-ac28-ec0a96dc9be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:01.965358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 870/1733] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:02.129380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:02.212950 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92f819-5e8f-4b01-a655-7e36d2d5b28b None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:02.330441 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831aded6-473d-4bb3-ac28-ec0a96dc9be3 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:02.365791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c10b4-27ee-46a0-94d5-892d3ebb365d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cedc7091d2f46b5be68818d26399e9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:02.367071 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c10b4-27ee-46a0-94d5-892d3ebb365d service swift] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:02.367603 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c10b4-27ee-46a0-94d5-892d3ebb365d service swift] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:02.368103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c10b4-27ee-46a0-94d5-892d3ebb365d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:02.548377 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92f819-5e8f-4b01-a655-7e36d2d5b28b None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:02.567811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b15f65-cbac-4f05-89bb-85883b33f679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:02.568932 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b15f65-cbac-4f05-89bb-85883b33f679 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:02.570688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b15f65-cbac-4f05-89bb-85883b33f679 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:02.571252 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b15f65-cbac-4f05-89bb-85883b33f679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:02.610082 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b15f65-cbac-4f05-89bb-85883b33f679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:02.617463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b15f65-cbac-4f05-89bb-85883b33f679 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:02.661970 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c10b4-27ee-46a0-94d5-892d3ebb365d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:02.665662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 871/1734] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:33:02.672397 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2cc2e-e722-4970-a9c6-83b8efc6bac7 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:02.672851 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2cc2e-e722-4970-a9c6-83b8efc6bac7 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:02.673350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2cc2e-e722-4970-a9c6-83b8efc6bac7 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:02.674044 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c10b4-27ee-46a0-94d5-892d3ebb365d service swift] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:02.681960 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 872/1735] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 7 13:33:02 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:33:02.699868 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 864/1736] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 7 13:33:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 886 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 13:33:02.741070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861ef09-2055-46e8-a382-db0c94c9c483 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:02.741570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861ef09-2055-46e8-a382-db0c94c9c483 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:02.742257 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861ef09-2055-46e8-a382-db0c94c9c483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:02.901422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-d861ef09-2055-46e8-a382-db0c94c9c483 None None] MFA Rules not processed for user `bee90bf951f04c2e8e19ceeacb8cfcb2`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:02.908550 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d861ef09-2055-46e8-a382-db0c94c9c483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:03.174771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c10b4-27ee-46a0-94d5-892d3ebb365d service swift] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:03.177386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-624c10b4-27ee-46a0-94d5-892d3ebb365d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:03.288317 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:03.326943 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c10b4-27ee-46a0-94d5-892d3ebb365d service swift] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:03.375942 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 873/1737] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 7 13:33:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 637 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:03.395374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a6c15-7d5c-4c91-bc37-dfa92263f252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:03.402903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a6c15-7d5c-4c91-bc37-dfa92263f252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:03.402903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a6c15-7d5c-4c91-bc37-dfa92263f252 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:03.402903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a6c15-7d5c-4c91-bc37-dfa92263f252 admin admin] PATH_INFO: `/v3/projects/c510ebda3c4144b29607b7fe17f271fc` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:03.404446 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a6c15-7d5c-4c91-bc37-dfa92263f252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c510ebda3c4144b29607b7fe17f271fc)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:03.419179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a6c15-7d5c-4c91-bc37-dfa92263f252 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:03.535535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861ef09-2055-46e8-a382-db0c94c9c483 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:03.539788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d861ef09-2055-46e8-a382-db0c94c9c483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:03.741603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861ef09-2055-46e8-a382-db0c94c9c483 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:03.787744 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da65808-c7d0-4aa8-92cc-ee6697887dab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bee90bf951f04c2e8e19ceeacb8cfcb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:03.789169 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da65808-c7d0-4aa8-92cc-ee6697887dab service placement] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:03.789834 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da65808-c7d0-4aa8-92cc-ee6697887dab service placement] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:03.790665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da65808-c7d0-4aa8-92cc-ee6697887dab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:03.857302 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da65808-c7d0-4aa8-92cc-ee6697887dab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:03.868091 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da65808-c7d0-4aa8-92cc-ee6697887dab service placement] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:03.909581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f70a6c15-7d5c-4c91-bc37-dfa92263f252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c510ebda3c4144b29607b7fe17f271fc', 'request_id': 'req-f70a6c15-7d5c-4c91-bc37-dfa92263f252'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:03.953523 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 874/1738] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:33:03.955639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da65808-c7d0-4aa8-92cc-ee6697887dab service placement] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:03.959829 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f70a6c15-7d5c-4c91-bc37-dfa92263f252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c510ebda3c4144b29607b7fe17f271fc', 'request_id': 'req-f70a6c15-7d5c-4c91-bc37-dfa92263f252'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:03.978645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1da65808-c7d0-4aa8-92cc-ee6697887dab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:04.131883 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da65808-c7d0-4aa8-92cc-ee6697887dab service placement] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:04.184785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f70a6c15-7d5c-4c91-bc37-dfa92263f252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c510ebda3c4144b29607b7fe17f271fc', 'request_id': 'req-f70a6c15-7d5c-4c91-bc37-dfa92263f252'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:04.256902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f70a6c15-7d5c-4c91-bc37-dfa92263f252 admin admin] The token cache is being invalidate because project c510ebda3c4144b29607b7fe17f271fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:33:04.256902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f70a6c15-7d5c-4c91-bc37-dfa92263f252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:04.256902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f70a6c15-7d5c-4c91-bc37-dfa92263f252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:04.272580 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 865/1739] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:33:03 2022] DELETE /identity/v3/projects/c510ebda3c4144b29607b7fe17f271fc => generated 0 bytes in 1092 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:04.284188 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad14f7-beb3-4f44-9b86-caf3c48bd7b0 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:04.284960 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad14f7-beb3-4f44-9b86-caf3c48bd7b0 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:04.285593 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad14f7-beb3-4f44-9b86-caf3c48bd7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:04.423776 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-3bad14f7-beb3-4f44-9b86-caf3c48bd7b0 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:04.426018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-3bad14f7-beb3-4f44-9b86-caf3c48bd7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:04.576218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bed6f-75aa-4a13-ad85-4856e067c2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:04.577241 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bed6f-75aa-4a13-ad85-4856e067c2f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:04.577643 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bed6f-75aa-4a13-ad85-4856e067c2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:04.577999 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bed6f-75aa-4a13-ad85-4856e067c2f9 admin admin] PATH_INFO: `/v3/users/cbea600cbfc74c069b457c4bba559287` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:04.579715 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bed6f-75aa-4a13-ad85-4856e067c2f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbea600cbfc74c069b457c4bba559287)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:04.586276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bed6f-75aa-4a13-ad85-4856e067c2f9 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:04.814516 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:04.841363 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 866/1740] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 568 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:04.844270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473b97b-509f-40d7-a1a2-9cdbc697fb36 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:04.845164 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473b97b-509f-40d7-a1a2-9cdbc697fb36 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:04.845955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473b97b-509f-40d7-a1a2-9cdbc697fb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:04.881257 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-0473b97b-509f-40d7-a1a2-9cdbc697fb36 None None] MFA Rules not processed for user `5acdbdb6f23b4f10bcbab0387ea67ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:04.882723 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-0473b97b-509f-40d7-a1a2-9cdbc697fb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:04.978150 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2e8bed6f-75aa-4a13-ad85-4856e067c2f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbea600cbfc74c069b457c4bba559287'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:05.091761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2e8bed6f-75aa-4a13-ad85-4856e067c2f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbea600cbfc74c069b457c4bba559287'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:05.092917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2e8bed6f-75aa-4a13-ad85-4856e067c2f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbea600cbfc74c069b457c4bba559287'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:05.094127 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2e8bed6f-75aa-4a13-ad85-4856e067c2f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbea600cbfc74c069b457c4bba559287'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:05.098007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 867/1741] 10.222.0.74 () {58 vars in 1015 bytes} [Wed Sep 7 13:33:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:05.110739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24d5a0-bf79-4b7f-921f-36346d36a912 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:05.111282 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24d5a0-bf79-4b7f-921f-36346d36a912 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:05.111768 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24d5a0-bf79-4b7f-921f-36346d36a912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:05.146280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2e8bed6f-75aa-4a13-ad85-4856e067c2f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbea600cbfc74c069b457c4bba559287'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:05.217904 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-4e24d5a0-bf79-4b7f-921f-36346d36a912 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:05.225234 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-4e24d5a0-bf79-4b7f-921f-36346d36a912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:05.296846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 868/1742] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:05.302613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24d5a0-bf79-4b7f-921f-36346d36a912 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:05.317894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-4e24d5a0-bf79-4b7f-921f-36346d36a912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:05.326073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 875/1743] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:33:03 2022] DELETE /identity/v3/users/cbea600cbfc74c069b457c4bba559287 => generated 0 bytes in 1364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:05.326073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bed6f-75aa-4a13-ad85-4856e067c2f9 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:05.413729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24d5a0-bf79-4b7f-921f-36346d36a912 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:05.624847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:05.635252 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bed6f-75aa-4a13-ad85-4856e067c2f9 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:05.656932 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2da0c9-f113-4e9a-ab02-f9011839eb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:05.661534 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2da0c9-f113-4e9a-ab02-f9011839eb6c admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:05.661534 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2da0c9-f113-4e9a-ab02-f9011839eb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:05.661534 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2da0c9-f113-4e9a-ab02-f9011839eb6c admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:05.661534 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2da0c9-f113-4e9a-ab02-f9011839eb6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:33:05.661534 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2da0c9-f113-4e9a-ab02-f9011839eb6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:05.665419 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2da0c9-f113-4e9a-ab02-f9011839eb6c admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:05.730828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7cac5-ab50-445f-9d14-87ae55ea20a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:05.732287 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7cac5-ab50-445f-9d14-87ae55ea20a6 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:05.732779 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7cac5-ab50-445f-9d14-87ae55ea20a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:05.733260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7cac5-ab50-445f-9d14-87ae55ea20a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:05.740728 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d6d7cac5-ab50-445f-9d14-87ae55ea20a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:05.747044 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 876/1744] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:33:05.752046 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2da0c9-f113-4e9a-ab02-f9011839eb6c admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:05.790693 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7cac5-ab50-445f-9d14-87ae55ea20a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:05.799334 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7cac5-ab50-445f-9d14-87ae55ea20a6 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:05.818133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2da0c9-f113-4e9a-ab02-f9011839eb6c admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:05.848460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 869/1745] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:33:05.859849 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282fcc49-eaea-4a3e-891e-24a9d3640288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:05.861154 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282fcc49-eaea-4a3e-891e-24a9d3640288 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:05.861815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282fcc49-eaea-4a3e-891e-24a9d3640288 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:05.863682 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282fcc49-eaea-4a3e-891e-24a9d3640288 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:05.868683 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282fcc49-eaea-4a3e-891e-24a9d3640288 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:05.879012 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282fcc49-eaea-4a3e-891e-24a9d3640288 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:06.063559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 877/1746] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:33:05 2022] POST /identity/v3/projects => generated 385 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:33:06.070489 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7cac5-ab50-445f-9d14-87ae55ea20a6 service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:06.151470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7cac5-ab50-445f-9d14-87ae55ea20a6 service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:06.433968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151a998-6a26-4bda-ab0f-1f2ac71694ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:06.436024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151a998-6a26-4bda-ab0f-1f2ac71694ee admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:06.436605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151a998-6a26-4bda-ab0f-1f2ac71694ee admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:06.437110 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151a998-6a26-4bda-ab0f-1f2ac71694ee admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:06.438161 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151a998-6a26-4bda-ab0f-1f2ac71694ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:06.441345 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151a998-6a26-4bda-ab0f-1f2ac71694ee admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:06.643425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 870/1747] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:33:06 2022] POST /identity/v3/users => generated 293 bytes in 575 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:33:06.651316 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282fcc49-eaea-4a3e-891e-24a9d3640288 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:06.690119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282fcc49-eaea-4a3e-891e-24a9d3640288 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:06.704823 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc077a79-ce23-438d-939e-79899b34ad11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:06.705873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc077a79-ce23-438d-939e-79899b34ad11 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:06.706369 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc077a79-ce23-438d-939e-79899b34ad11 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:06.706873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc077a79-ce23-438d-939e-79899b34ad11 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:06.708242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc077a79-ce23-438d-939e-79899b34ad11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:06.711660 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc077a79-ce23-438d-939e-79899b34ad11 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:06.767904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 878/1748] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:33:06.769507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151a998-6a26-4bda-ab0f-1f2ac71694ee admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:06.791855 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151a998-6a26-4bda-ab0f-1f2ac71694ee admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:06.820216 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0edf95-ee06-41be-9e11-6b5fde2b3db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:06.820216 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0edf95-ee06-41be-9e11-6b5fde2b3db5 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:06.820216 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0edf95-ee06-41be-9e11-6b5fde2b3db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:06.820216 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0edf95-ee06-41be-9e11-6b5fde2b3db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:06.821676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0edf95-ee06-41be-9e11-6b5fde2b3db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:06.852414 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0edf95-ee06-41be-9e11-6b5fde2b3db5 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:06.878164 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 871/1749] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:33:06.883764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc077a79-ce23-438d-939e-79899b34ad11 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:06.906797 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc077a79-ce23-438d-939e-79899b34ad11 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:06.962238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea685f29-3d76-4624-9cd9-bd0cc45a1b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:06.963619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea685f29-3d76-4624-9cd9-bd0cc45a1b42 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:06.964188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea685f29-3d76-4624-9cd9-bd0cc45a1b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:06.965364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea685f29-3d76-4624-9cd9-bd0cc45a1b42 admin admin] PATH_INFO: `/v3/projects/7be5c62f81284249ab09bf6c72fc65ca/users/0c0bf538904f4d39bed7f29ab29bed1e/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:07.032327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea685f29-3d76-4624-9cd9-bd0cc45a1b42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7be5c62f81284249ab09bf6c72fc65ca, user_id=0c0bf538904f4d39bed7f29ab29bed1e, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:07.052979 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea685f29-3d76-4624-9cd9-bd0cc45a1b42 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:07.211559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 879/1750] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:33:06 2022] PUT /identity/v3/projects/7be5c62f81284249ab09bf6c72fc65ca/users/0c0bf538904f4d39bed7f29ab29bed1e/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:07.225492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e18f6d-2c0f-48f6-87e6-3ee04c4fe683 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:07.227867 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e18f6d-2c0f-48f6-87e6-3ee04c4fe683 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:07.227867 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e18f6d-2c0f-48f6-87e6-3ee04c4fe683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:07.369041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-86e18f6d-2c0f-48f6-87e6-3ee04c4fe683 None None] MFA Rules not processed for user `0c0bf538904f4d39bed7f29ab29bed1e`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:07.374083 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-86e18f6d-2c0f-48f6-87e6-3ee04c4fe683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:07.762848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:07.778458 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 872/1751] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 553 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:07.792884 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea685f29-3d76-4624-9cd9-bd0cc45a1b42 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:08.096624 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea685f29-3d76-4624-9cd9-bd0cc45a1b42 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:08.138911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e40d26-46a7-441d-b810-438fe3ae085a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:08.140883 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e40d26-46a7-441d-b810-438fe3ae085a admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:08.141420 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e40d26-46a7-441d-b810-438fe3ae085a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:08.141845 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e40d26-46a7-441d-b810-438fe3ae085a admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:08.142910 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e40d26-46a7-441d-b810-438fe3ae085a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:08.154104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e40d26-46a7-441d-b810-438fe3ae085a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:08.453363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 880/1752] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:33:07 2022] POST /identity/v3/projects => generated 387 bytes in 669 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:33:08.463696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e18f6d-2c0f-48f6-87e6-3ee04c4fe683 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:08.538694 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e18f6d-2c0f-48f6-87e6-3ee04c4fe683 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:08.755912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e40d26-46a7-441d-b810-438fe3ae085a admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:08.759946 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f6e40d26-46a7-441d-b810-438fe3ae085a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:08.918895 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76880db8-874a-4017-ac5d-c1871273ba93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:08.921522 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76880db8-874a-4017-ac5d-c1871273ba93 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:08.922273 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76880db8-874a-4017-ac5d-c1871273ba93 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:08.923020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76880db8-874a-4017-ac5d-c1871273ba93 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:08.932289 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76880db8-874a-4017-ac5d-c1871273ba93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:08.938477 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76880db8-874a-4017-ac5d-c1871273ba93 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:09.195285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e40d26-46a7-441d-b810-438fe3ae085a admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:09.219286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be368c28-240c-488c-9a6a-9f704c57b4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:09.221048 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be368c28-240c-488c-9a6a-9f704c57b4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:09.221940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be368c28-240c-488c-9a6a-9f704c57b4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:09.222719 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be368c28-240c-488c-9a6a-9f704c57b4ba admin admin] PATH_INFO: `/v3/projects/3a808904f29044cc8bb01573c860702d` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:09.228550 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be368c28-240c-488c-9a6a-9f704c57b4ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a808904f29044cc8bb01573c860702d)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:09.233618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be368c28-240c-488c-9a6a-9f704c57b4ba admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:09.278427 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 873/1753] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:33:08 2022] POST /identity/v3/users => generated 294 bytes in 818 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:33:09.285493 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76880db8-874a-4017-ac5d-c1871273ba93 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:09.305031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76880db8-874a-4017-ac5d-c1871273ba93 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:09.315367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec62ee4-d28a-4d4e-bd16-f92706ed19dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:09.317114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec62ee4-d28a-4d4e-bd16-f92706ed19dc admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:09.317715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec62ee4-d28a-4d4e-bd16-f92706ed19dc admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:09.318934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec62ee4-d28a-4d4e-bd16-f92706ed19dc admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:09.320901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec62ee4-d28a-4d4e-bd16-f92706ed19dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:09.330100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec62ee4-d28a-4d4e-bd16-f92706ed19dc admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:09.340172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:09.351406 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 874/1754] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:33:09.362999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec62ee4-d28a-4d4e-bd16-f92706ed19dc admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:09.387945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec62ee4-d28a-4d4e-bd16-f92706ed19dc admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:09.459794 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72986136-094d-431b-bf32-4510a49f1ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:09.463918 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72986136-094d-431b-bf32-4510a49f1ddb admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:09.464458 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72986136-094d-431b-bf32-4510a49f1ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:09.464972 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72986136-094d-431b-bf32-4510a49f1ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:09.466321 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72986136-094d-431b-bf32-4510a49f1ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:09.470324 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72986136-094d-431b-bf32-4510a49f1ddb admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:09.536853 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 875/1755] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:33:09.550505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72986136-094d-431b-bf32-4510a49f1ddb admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:09.617676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72986136-094d-431b-bf32-4510a49f1ddb admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:09.635521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-be368c28-240c-488c-9a6a-9f704c57b4ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a808904f29044cc8bb01573c860702d', 'request_id': 'req-be368c28-240c-488c-9a6a-9f704c57b4ba'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:09.676919 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-be368c28-240c-488c-9a6a-9f704c57b4ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a808904f29044cc8bb01573c860702d', 'request_id': 'req-be368c28-240c-488c-9a6a-9f704c57b4ba'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:09.816149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-be368c28-240c-488c-9a6a-9f704c57b4ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a808904f29044cc8bb01573c860702d', 'request_id': 'req-be368c28-240c-488c-9a6a-9f704c57b4ba'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:09.899497 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-be368c28-240c-488c-9a6a-9f704c57b4ba admin admin] The token cache is being invalidate because project 3a808904f29044cc8bb01573c860702d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:33:09.904596 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-be368c28-240c-488c-9a6a-9f704c57b4ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:09.911163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-be368c28-240c-488c-9a6a-9f704c57b4ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:09.927097 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 881/1756] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:33:08 2022] DELETE /identity/v3/projects/3a808904f29044cc8bb01573c860702d => generated 0 bytes in 1172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:09.939081 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be368c28-240c-488c-9a6a-9f704c57b4ba admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:09.951826 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-be368c28-240c-488c-9a6a-9f704c57b4ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:10.037428 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01bf89-e8e6-407c-ab68-d75bfb8adf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:10.038355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01bf89-e8e6-407c-ab68-d75bfb8adf55 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:10.043445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01bf89-e8e6-407c-ab68-d75bfb8adf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:10.043445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01bf89-e8e6-407c-ab68-d75bfb8adf55 admin admin] PATH_INFO: `/v3/projects/3f339dfb911c441d80aab6fe7ba62095/users/286cbafca3804eb38646677b739c486f/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:10.167395 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01bf89-e8e6-407c-ab68-d75bfb8adf55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f339dfb911c441d80aab6fe7ba62095, user_id=286cbafca3804eb38646677b739c486f, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:10.197571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01bf89-e8e6-407c-ab68-d75bfb8adf55 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:10.341485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:10.382179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 876/1757] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:33:09 2022] PUT /identity/v3/projects/3f339dfb911c441d80aab6fe7ba62095/users/286cbafca3804eb38646677b739c486f/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 826 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:10.398705 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae48b7-89b9-4e51-a767-3ce8aada132c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:10.399042 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae48b7-89b9-4e51-a767-3ce8aada132c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:10.399205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae48b7-89b9-4e51-a767-3ce8aada132c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:10.456335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be368c28-240c-488c-9a6a-9f704c57b4ba admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:10.597643 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-23ae48b7-89b9-4e51-a767-3ce8aada132c None None] MFA Rules not processed for user `286cbafca3804eb38646677b739c486f`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:10.602471 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-23ae48b7-89b9-4e51-a767-3ce8aada132c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:10.810831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cd0af-f0b5-4c02-9007-aaf4c2522d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:10.812004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cd0af-f0b5-4c02-9007-aaf4c2522d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:10.812494 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cd0af-f0b5-4c02-9007-aaf4c2522d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:10.813051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cd0af-f0b5-4c02-9007-aaf4c2522d13 admin admin] PATH_INFO: `/v3/users/7f8c7dc2115b44128d800086dee7cfe4` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:10.828406 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cd0af-f0b5-4c02-9007-aaf4c2522d13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f8c7dc2115b44128d800086dee7cfe4)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:10.842309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cd0af-f0b5-4c02-9007-aaf4c2522d13 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:11.040586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 877/1758] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 643 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:11.044565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae48b7-89b9-4e51-a767-3ce8aada132c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:11.051812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-23ae48b7-89b9-4e51-a767-3ce8aada132c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:11.169618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae48b7-89b9-4e51-a767-3ce8aada132c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:11.191501 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d219ed0-be6b-4929-8f2e-a729cc2f7d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:11.209872 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d219ed0-be6b-4929-8f2e-a729cc2f7d75 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:11.213269 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d219ed0-be6b-4929-8f2e-a729cc2f7d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:11.217975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d219ed0-be6b-4929-8f2e-a729cc2f7d75 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:11.221395 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d219ed0-be6b-4929-8f2e-a729cc2f7d75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:11.259170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d219ed0-be6b-4929-8f2e-a729cc2f7d75 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:11.349991 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e25cd0af-f0b5-4c02-9007-aaf4c2522d13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f8c7dc2115b44128d800086dee7cfe4'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:11.476167 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e25cd0af-f0b5-4c02-9007-aaf4c2522d13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f8c7dc2115b44128d800086dee7cfe4'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:11.482302 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e25cd0af-f0b5-4c02-9007-aaf4c2522d13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f8c7dc2115b44128d800086dee7cfe4'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:11.485645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e25cd0af-f0b5-4c02-9007-aaf4c2522d13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f8c7dc2115b44128d800086dee7cfe4'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:11.490811 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 878/1759] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:33:11 2022] POST /identity/v3/projects => generated 387 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:33:11.497258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d219ed0-be6b-4929-8f2e-a729cc2f7d75 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:11.499401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-3d219ed0-be6b-4929-8f2e-a729cc2f7d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:11.521888 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:11.545043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e25cd0af-f0b5-4c02-9007-aaf4c2522d13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f8c7dc2115b44128d800086dee7cfe4'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:11.749821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 882/1760] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:33:09 2022] DELETE /identity/v3/users/7f8c7dc2115b44128d800086dee7cfe4 => generated 0 bytes in 1812 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:11.832899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d219ed0-be6b-4929-8f2e-a729cc2f7d75 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:12.077879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263a3e2-4e69-4c4b-8802-33321a2fc61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:12.079619 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263a3e2-4e69-4c4b-8802-33321a2fc61d admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:12.080186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263a3e2-4e69-4c4b-8802-33321a2fc61d admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:12.080835 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263a3e2-4e69-4c4b-8802-33321a2fc61d admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:12.083515 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263a3e2-4e69-4c4b-8802-33321a2fc61d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:12.087656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263a3e2-4e69-4c4b-8802-33321a2fc61d admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:12.193553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffb446-190c-477b-b212-8c5f52bc683c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:12.193553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffb446-190c-477b-b212-8c5f52bc683c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:12.194251 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffb446-190c-477b-b212-8c5f52bc683c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:12.307605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 879/1761] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:33:11 2022] POST /identity/v3/users => generated 300 bytes in 811 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:33:12.314572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263a3e2-4e69-4c4b-8802-33321a2fc61d admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:12.352815 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:12.372835 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263a3e2-4e69-4c4b-8802-33321a2fc61d admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:12.391562 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2501ba5-2ae2-4974-9afc-ea543455c42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:12.397210 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2501ba5-2ae2-4974-9afc-ea543455c42c admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:12.397414 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-f3ffb446-190c-477b-b212-8c5f52bc683c None None] MFA Rules not processed for user `32401931d75649018f6b0e55144bb5f2`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:12.397691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2501ba5-2ae2-4974-9afc-ea543455c42c admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:12.398176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2501ba5-2ae2-4974-9afc-ea543455c42c admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:12.402546 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f3ffb446-190c-477b-b212-8c5f52bc683c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:12.405542 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2501ba5-2ae2-4974-9afc-ea543455c42c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:12.418826 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2501ba5-2ae2-4974-9afc-ea543455c42c admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:12.482044 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 880/1762] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:33:12.486756 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2501ba5-2ae2-4974-9afc-ea543455c42c admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:12.539563 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2501ba5-2ae2-4974-9afc-ea543455c42c admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:12.556635 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8a551-a858-4060-9d7c-4a4b702c9841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:12.557838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8a551-a858-4060-9d7c-4a4b702c9841 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:12.558888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8a551-a858-4060-9d7c-4a4b702c9841 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:12.559429 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8a551-a858-4060-9d7c-4a4b702c9841 admin admin] PATH_INFO: `/v3/projects/5afc0be3c2174c7d95c146a414a06696/users/1d9551e37ad24a20bb1d6723bf512124/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:12.611047 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 883/1763] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:33:12.617823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8a551-a858-4060-9d7c-4a4b702c9841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afc0be3c2174c7d95c146a414a06696, user_id=1d9551e37ad24a20bb1d6723bf512124, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:12.624676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8a551-a858-4060-9d7c-4a4b702c9841 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:12.643458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffb446-190c-477b-b212-8c5f52bc683c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:12.649631 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f3ffb446-190c-477b-b212-8c5f52bc683c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:12.741992 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 881/1764] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:33:12 2022] PUT /identity/v3/projects/5afc0be3c2174c7d95c146a414a06696/users/1d9551e37ad24a20bb1d6723bf512124/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:12.749829 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8a551-a858-4060-9d7c-4a4b702c9841 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:13.100827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8a551-a858-4060-9d7c-4a4b702c9841 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:13.122873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af0b9e-63e8-4e0d-a219-9265cc20441f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:13.124015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af0b9e-63e8-4e0d-a219-9265cc20441f admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:13.128304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af0b9e-63e8-4e0d-a219-9265cc20441f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:13.128968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af0b9e-63e8-4e0d-a219-9265cc20441f admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:13.133188 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af0b9e-63e8-4e0d-a219-9265cc20441f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:13.136223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af0b9e-63e8-4e0d-a219-9265cc20441f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:13.148476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffb446-190c-477b-b212-8c5f52bc683c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:13.228700 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 882/1765] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:33:13.229400 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadeb9c-2ff0-4f24-8e20-3f7166a0f271 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:13.229912 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadeb9c-2ff0-4f24-8e20-3f7166a0f271 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:13.232573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadeb9c-2ff0-4f24-8e20-3f7166a0f271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:13.341263 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:13.349812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-ceadeb9c-2ff0-4f24-8e20-3f7166a0f271 None None] MFA Rules not processed for user `3a9a1057fff246d7b41234c073cdb914`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:13.357745 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ceadeb9c-2ff0-4f24-8e20-3f7166a0f271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:13.362392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9a46c-b1fd-46f5-83a3-056d410bba1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:13.363966 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9a46c-b1fd-46f5-83a3-056d410bba1b service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:13.367141 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9a46c-b1fd-46f5-83a3-056d410bba1b service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:13.367141 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9a46c-b1fd-46f5-83a3-056d410bba1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:13.563200 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b9a46c-b1fd-46f5-83a3-056d410bba1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:13.572253 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b9a46c-b1fd-46f5-83a3-056d410bba1b service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:13.621140 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 884/1766] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 980 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:33:13.628043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b0592-881e-492b-8449-7a016b28d4e0 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:13.639539 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b0592-881e-492b-8449-7a016b28d4e0 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:13.642559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b0592-881e-492b-8449-7a016b28d4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:13.742666 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 883/1767] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 517 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:13.746307 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadeb9c-2ff0-4f24-8e20-3f7166a0f271 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:13.755114 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-f98b0592-881e-492b-8449-7a016b28d4e0 None None] MFA Rules not processed for user `7e7db29f64934026abbe96d5cb5baf25`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:13.762405 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f98b0592-881e-492b-8449-7a016b28d4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:13.783560 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadeb9c-2ff0-4f24-8e20-3f7166a0f271 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:13.805200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf08259-f794-4ed6-8a43-9c0bc6491618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:13.806112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf08259-f794-4ed6-8a43-9c0bc6491618 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:13.806853 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf08259-f794-4ed6-8a43-9c0bc6491618 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:13.809659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf08259-f794-4ed6-8a43-9c0bc6491618 admin admin] PATH_INFO: `/v3/projects/5afc0be3c2174c7d95c146a414a06696/users/1d9551e37ad24a20bb1d6723bf512124/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:13.823010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf08259-f794-4ed6-8a43-9c0bc6491618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afc0be3c2174c7d95c146a414a06696, user_id=1d9551e37ad24a20bb1d6723bf512124, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:13.835924 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf08259-f794-4ed6-8a43-9c0bc6491618 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:13.888641 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 884/1768] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:33:13 2022] PUT /identity/v3/projects/5afc0be3c2174c7d95c146a414a06696/users/1d9551e37ad24a20bb1d6723bf512124/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:13.890199 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf08259-f794-4ed6-8a43-9c0bc6491618 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:13.897462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-bdf08259-f794-4ed6-8a43-9c0bc6491618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:13.991944 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 885/1769] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:13.994523 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ed36a-f810-48b7-aee3-4a07ffcede71 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:13.994875 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ed36a-f810-48b7-aee3-4a07ffcede71 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:13.995254 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ed36a-f810-48b7-aee3-4a07ffcede71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:14.061523 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:14.149424 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-aa2ed36a-f810-48b7-aee3-4a07ffcede71 None None] MFA Rules not processed for user `1d9551e37ad24a20bb1d6723bf512124`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:14.154719 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-aa2ed36a-f810-48b7-aee3-4a07ffcede71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:14.177258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf08259-f794-4ed6-8a43-9c0bc6491618 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:14.207044 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606ffb6-5c62-432e-ad01-00971d4dc13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:14.208163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606ffb6-5c62-432e-ad01-00971d4dc13c admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:14.208572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606ffb6-5c62-432e-ad01-00971d4dc13c admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:14.209130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606ffb6-5c62-432e-ad01-00971d4dc13c admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:14.210416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2606ffb6-5c62-432e-ad01-00971d4dc13c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:14.220539 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2606ffb6-5c62-432e-ad01-00971d4dc13c admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:14.471717 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 885/1770] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:33:13 2022] POST /identity/v3/projects => generated 371 bytes in 577 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:33:14.494950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606ffb6-5c62-432e-ad01-00971d4dc13c admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:14.509849 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2606ffb6-5c62-432e-ad01-00971d4dc13c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:14.514798 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 886/1771] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 522 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:14.519930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ed36a-f810-48b7-aee3-4a07ffcede71 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:14.557638 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ed36a-f810-48b7-aee3-4a07ffcede71 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:14.631502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:14.774887 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d651ab-33df-4877-81de-213619f85ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:14.774887 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d651ab-33df-4877-81de-213619f85ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:14.774887 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d651ab-33df-4877-81de-213619f85ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:14.774887 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d651ab-33df-4877-81de-213619f85ef9 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:14.775484 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d651ab-33df-4877-81de-213619f85ef9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:14.784570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d651ab-33df-4877-81de-213619f85ef9 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:14.976417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606ffb6-5c62-432e-ad01-00971d4dc13c admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:14.993742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 887/1772] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:33:14 2022] POST /identity/v3/users => generated 292 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:33:15.000353 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d651ab-33df-4877-81de-213619f85ef9 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:15.004380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d386000-7b5f-459f-a789-826ea0eca51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:15.005617 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d386000-7b5f-459f-a789-826ea0eca51f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:15.005948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d386000-7b5f-459f-a789-826ea0eca51f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:15.006409 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d386000-7b5f-459f-a789-826ea0eca51f admin admin] PATH_INFO: `/v3/projects/3badd2f1a6444efd97043e547bc60930` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:15.026129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d386000-7b5f-459f-a789-826ea0eca51f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3badd2f1a6444efd97043e547bc60930)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:15.029949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d386000-7b5f-459f-a789-826ea0eca51f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:15.032470 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f6d651ab-33df-4877-81de-213619f85ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:15.357448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d651ab-33df-4877-81de-213619f85ef9 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:15.421233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5d386000-7b5f-459f-a789-826ea0eca51f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3badd2f1a6444efd97043e547bc60930', 'request_id': 'req-5d386000-7b5f-459f-a789-826ea0eca51f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:15.481347 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5d386000-7b5f-459f-a789-826ea0eca51f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3badd2f1a6444efd97043e547bc60930', 'request_id': 'req-5d386000-7b5f-459f-a789-826ea0eca51f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:15.587742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5d386000-7b5f-459f-a789-826ea0eca51f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3badd2f1a6444efd97043e547bc60930', 'request_id': 'req-5d386000-7b5f-459f-a789-826ea0eca51f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:15.587905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd947647-7ac3-471a-bdf3-afc447a26868 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:15.587905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd947647-7ac3-471a-bdf3-afc447a26868 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:15.587905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd947647-7ac3-471a-bdf3-afc447a26868 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:15.587905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd947647-7ac3-471a-bdf3-afc447a26868 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:15.622142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5d386000-7b5f-459f-a789-826ea0eca51f admin admin] The token cache is being invalidate because project 3badd2f1a6444efd97043e547bc60930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:33:15.623486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5d386000-7b5f-459f-a789-826ea0eca51f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:15.624882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5d386000-7b5f-459f-a789-826ea0eca51f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:15.626857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 886/1773] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:33:14 2022] DELETE /identity/v3/projects/3badd2f1a6444efd97043e547bc60930 => generated 0 bytes in 1160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:15.628318 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d386000-7b5f-459f-a789-826ea0eca51f admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:15.630148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5d386000-7b5f-459f-a789-826ea0eca51f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:15.765674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:15.789469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d386000-7b5f-459f-a789-826ea0eca51f admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:15.816003 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e390bd-c059-4a9a-bef0-6c1ff2a905c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:15.819707 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e390bd-c059-4a9a-bef0-6c1ff2a905c7 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:15.847536 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e390bd-c059-4a9a-bef0-6c1ff2a905c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:15.847536 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e390bd-c059-4a9a-bef0-6c1ff2a905c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:15.847536 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e390bd-c059-4a9a-bef0-6c1ff2a905c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:15.847536 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e390bd-c059-4a9a-bef0-6c1ff2a905c7 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:15.848063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd947647-7ac3-471a-bdf3-afc447a26868 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:15.852746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd947647-7ac3-471a-bdf3-afc447a26868 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:15.876987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 887/1774] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:33:15.876987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626867e3-9cb5-46a3-8695-c4742cc71f68 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:15.916161 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626867e3-9cb5-46a3-8695-c4742cc71f68 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:15.916161 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626867e3-9cb5-46a3-8695-c4742cc71f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:15.918047 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 888/1775] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 893 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:33:15.918047 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd947647-7ac3-471a-bdf3-afc447a26868 service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:15.965628 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-626867e3-9cb5-46a3-8695-c4742cc71f68 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:15.967387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd947647-7ac3-471a-bdf3-afc447a26868 service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:15.967982 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-626867e3-9cb5-46a3-8695-c4742cc71f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:16.009829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda8739-ae6a-4a48-a269-2316e47508a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:16.014207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda8739-ae6a-4a48-a269-2316e47508a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:16.023519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda8739-ae6a-4a48-a269-2316e47508a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:16.023519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda8739-ae6a-4a48-a269-2316e47508a0 admin admin] PATH_INFO: `/v3/projects/55dfc66bdba247998846b11a4cbabe64/users/7e5f1078006549c4b93a175f733b5b3a/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:16.024042 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 888/1776] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:16.029312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc5a2f-7751-464f-bef2-925ac2c95394 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:16.047540 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc5a2f-7751-464f-bef2-925ac2c95394 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:16.060903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc5a2f-7751-464f-bef2-925ac2c95394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:16.168105 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda8739-ae6a-4a48-a269-2316e47508a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55dfc66bdba247998846b11a4cbabe64, user_id=7e5f1078006549c4b93a175f733b5b3a, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:16.200626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda8739-ae6a-4a48-a269-2316e47508a0 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:16.273141 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-93dc5a2f-7751-464f-bef2-925ac2c95394 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:16.283696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-93dc5a2f-7751-464f-bef2-925ac2c95394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:16.371744 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 889/1777] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:16.376921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc5a2f-7751-464f-bef2-925ac2c95394 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:16.387801 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 889/1778] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:33:15 2022] PUT /identity/v3/projects/55dfc66bdba247998846b11a4cbabe64/users/7e5f1078006549c4b93a175f733b5b3a/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:16.394446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda8739-ae6a-4a48-a269-2316e47508a0 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:16.406266 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:16.483169 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc5a2f-7751-464f-bef2-925ac2c95394 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:16.841979 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda8739-ae6a-4a48-a269-2316e47508a0 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:16.882387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf678a0-a1d0-403c-86db-ad2092b34e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:16.884672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf678a0-a1d0-403c-86db-ad2092b34e52 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:16.888148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf678a0-a1d0-403c-86db-ad2092b34e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:16.888926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf678a0-a1d0-403c-86db-ad2092b34e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:16.891749 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf678a0-a1d0-403c-86db-ad2092b34e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:16.902349 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5b736-a34f-4b19-b5a4-d4d9831d9857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:16.904184 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf678a0-a1d0-403c-86db-ad2092b34e52 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:16.905871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5b736-a34f-4b19-b5a4-d4d9831d9857 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:16.909366 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5b736-a34f-4b19-b5a4-d4d9831d9857 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:16.912019 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5b736-a34f-4b19-b5a4-d4d9831d9857 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:16.913879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5b736-a34f-4b19-b5a4-d4d9831d9857 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:33:16.920090 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5b736-a34f-4b19-b5a4-d4d9831d9857 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:16.933721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5b736-a34f-4b19-b5a4-d4d9831d9857 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:16.983543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 890/1779] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 611 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:33:16.986750 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c26df5-9de6-44e6-a06f-0f0c2fbc221e None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:16.987997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c26df5-9de6-44e6-a06f-0f0c2fbc221e None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:16.988870 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c26df5-9de6-44e6-a06f-0f0c2fbc221e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:17.000782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 890/1780] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 608 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:33:17.002303 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf678a0-a1d0-403c-86db-ad2092b34e52 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:17.033489 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-e2c26df5-9de6-44e6-a06f-0f0c2fbc221e None None] MFA Rules not processed for user `c5e931aa22344990a0f032159ebc20ad`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:17.036684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e2c26df5-9de6-44e6-a06f-0f0c2fbc221e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:17.049329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf678a0-a1d0-403c-86db-ad2092b34e52 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:17.073349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c09624-4ab2-4577-b62d-a5453e490075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:17.074954 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c09624-4ab2-4577-b62d-a5453e490075 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:17.079443 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c09624-4ab2-4577-b62d-a5453e490075 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:17.079940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c09624-4ab2-4577-b62d-a5453e490075 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:17.084197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c09624-4ab2-4577-b62d-a5453e490075 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:17.094609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c09624-4ab2-4577-b62d-a5453e490075 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:17.247159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 891/1781] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:17.249438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c26df5-9de6-44e6-a06f-0f0c2fbc221e None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:17.274088 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c26df5-9de6-44e6-a06f-0f0c2fbc221e None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:17.284925 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f2f9f-6440-462d-84bc-9efed0978a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:17.286769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f2f9f-6440-462d-84bc-9efed0978a5f admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:17.287927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f2f9f-6440-462d-84bc-9efed0978a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:17.289002 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f2f9f-6440-462d-84bc-9efed0978a5f admin admin] PATH_INFO: `/v3/projects/55dfc66bdba247998846b11a4cbabe64/users/7e5f1078006549c4b93a175f733b5b3a/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:17.294211 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f2f9f-6440-462d-84bc-9efed0978a5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=55dfc66bdba247998846b11a4cbabe64, user_id=7e5f1078006549c4b93a175f733b5b3a, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:17.300888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f2f9f-6440-462d-84bc-9efed0978a5f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:17.326490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 892/1782] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:33:17 2022] PUT /identity/v3/projects/55dfc66bdba247998846b11a4cbabe64/users/7e5f1078006549c4b93a175f733b5b3a/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:17.332759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 891/1783] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:33:17 2022] POST /identity/v3/projects => generated 369 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:33:17.343756 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f2f9f-6440-462d-84bc-9efed0978a5f admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:17.739391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f2f9f-6440-462d-84bc-9efed0978a5f admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:17.791766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:18.033459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65326b52-18f9-4e66-ab48-1551ff50c57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:18.034594 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65326b52-18f9-4e66-ab48-1551ff50c57b admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:18.035079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65326b52-18f9-4e66-ab48-1551ff50c57b admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:18.035353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65326b52-18f9-4e66-ab48-1551ff50c57b admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:18.036063 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65326b52-18f9-4e66-ab48-1551ff50c57b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:18.038243 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65326b52-18f9-4e66-ab48-1551ff50c57b admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:18.234650 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 893/1784] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:33:17 2022] POST /identity/v3/users => generated 285 bytes in 892 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:33:18.241387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c09624-4ab2-4577-b62d-a5453e490075 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:18.313343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c09624-4ab2-4577-b62d-a5453e490075 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:18.335335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b135c-3594-467e-857f-d49e41087472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:18.336930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b135c-3594-467e-857f-d49e41087472 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:18.337674 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b135c-3594-467e-857f-d49e41087472 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:18.340552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b135c-3594-467e-857f-d49e41087472 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:18.343194 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b135c-3594-467e-857f-d49e41087472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:18.358582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b135c-3594-467e-857f-d49e41087472 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:18.418783 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65326b52-18f9-4e66-ab48-1551ff50c57b admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:18.422330 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 892/1785] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:33:18.446053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65326b52-18f9-4e66-ab48-1551ff50c57b admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:18.467023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41604f6c-7695-45fa-9140-f3c14efeb04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:18.467023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41604f6c-7695-45fa-9140-f3c14efeb04c admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:18.467023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41604f6c-7695-45fa-9140-f3c14efeb04c admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:18.467023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41604f6c-7695-45fa-9140-f3c14efeb04c admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:18.467674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41604f6c-7695-45fa-9140-f3c14efeb04c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:18.479139 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41604f6c-7695-45fa-9140-f3c14efeb04c admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:18.523260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b135c-3594-467e-857f-d49e41087472 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:18.535520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 894/1786] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:33:18.550726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b135c-3594-467e-857f-d49e41087472 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:18.587069 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a9955-9f0a-4be7-9022-1df5b7ceb7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:18.589730 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a9955-9f0a-4be7-9022-1df5b7ceb7a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:18.592765 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a9955-9f0a-4be7-9022-1df5b7ceb7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:18.594584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a9955-9f0a-4be7-9022-1df5b7ceb7a8 admin admin] PATH_INFO: `/v3/projects/f91810e2c6fb4ff3bd8b3db8a796e7ae/users/4e82898a6f044abea6d099e784b5c494/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:18.650472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a9955-9f0a-4be7-9022-1df5b7ceb7a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f91810e2c6fb4ff3bd8b3db8a796e7ae, user_id=4e82898a6f044abea6d099e784b5c494, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:18.664685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a9955-9f0a-4be7-9022-1df5b7ceb7a8 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:18.798213 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 893/1787] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:33:18 2022] PUT /identity/v3/projects/f91810e2c6fb4ff3bd8b3db8a796e7ae/users/4e82898a6f044abea6d099e784b5c494/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:18.813166 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd3281-07a1-4c22-8d18-740f5f66dcfd None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:18.813865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd3281-07a1-4c22-8d18-740f5f66dcfd None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:18.814369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd3281-07a1-4c22-8d18-740f5f66dcfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:18.926772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-b9bd3281-07a1-4c22-8d18-740f5f66dcfd None None] MFA Rules not processed for user `4e82898a6f044abea6d099e784b5c494`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:18.928450 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b9bd3281-07a1-4c22-8d18-740f5f66dcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:19.283788 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 895/1788] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:19.291493 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a9955-9f0a-4be7-9022-1df5b7ceb7a8 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:19.418215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd3281-07a1-4c22-8d18-740f5f66dcfd None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:19.445409 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b9bd3281-07a1-4c22-8d18-740f5f66dcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:19.517863 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a9955-9f0a-4be7-9022-1df5b7ceb7a8 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:19.540510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824ae6c-c511-4e6c-a945-812c977e0c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:19.541203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824ae6c-c511-4e6c-a945-812c977e0c00 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:19.541505 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824ae6c-c511-4e6c-a945-812c977e0c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:19.541928 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824ae6c-c511-4e6c-a945-812c977e0c00 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:19.543572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824ae6c-c511-4e6c-a945-812c977e0c00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:19.549555 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824ae6c-c511-4e6c-a945-812c977e0c00 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:19.736217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd3281-07a1-4c22-8d18-740f5f66dcfd None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:19.764735 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 894/1789] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:33:19 2022] POST /identity/v3/projects => generated 369 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:33:19.765906 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824ae6c-c511-4e6c-a945-812c977e0c00 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:19.890023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:19.894735 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:19.950952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad493d0f-e84d-4033-85c8-f4c6cbc8907f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:19.952851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad493d0f-e84d-4033-85c8-f4c6cbc8907f service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:19.953234 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad493d0f-e84d-4033-85c8-f4c6cbc8907f service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:19.953539 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad493d0f-e84d-4033-85c8-f4c6cbc8907f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:19.957618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ad493d0f-e84d-4033-85c8-f4c6cbc8907f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:20.079188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824ae6c-c511-4e6c-a945-812c977e0c00 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:20.088666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c739f-e82c-46f5-b6c2-51ee6d6004f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:20.090600 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c739f-e82c-46f5-b6c2-51ee6d6004f8 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:20.091230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c739f-e82c-46f5-b6c2-51ee6d6004f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:20.091804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c739f-e82c-46f5-b6c2-51ee6d6004f8 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:20.093113 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c739f-e82c-46f5-b6c2-51ee6d6004f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:20.111787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c739f-e82c-46f5-b6c2-51ee6d6004f8 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:20.227466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad493d0f-e84d-4033-85c8-f4c6cbc8907f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:20.245922 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad493d0f-e84d-4033-85c8-f4c6cbc8907f service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:20.312896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 895/1790] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:33:19 2022] POST /identity/v3/users => generated 291 bytes in 542 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:33:20.315987 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c739f-e82c-46f5-b6c2-51ee6d6004f8 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:20.324123 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 896/1791] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 906 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:33:20.372974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c739f-e82c-46f5-b6c2-51ee6d6004f8 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:20.410949 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3f166-401a-4b65-ab21-51e86c9817ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:20.412111 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3f166-401a-4b65-ab21-51e86c9817ad admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:20.412608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3f166-401a-4b65-ab21-51e86c9817ad admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:20.413096 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3f166-401a-4b65-ab21-51e86c9817ad admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:20.414486 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3f166-401a-4b65-ab21-51e86c9817ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:20.436292 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3f166-401a-4b65-ab21-51e86c9817ad admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:20.493974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 896/1792] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:33:20.500753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad493d0f-e84d-4033-85c8-f4c6cbc8907f service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:20.595018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad493d0f-e84d-4033-85c8-f4c6cbc8907f service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:20.612462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23ee1c-6d64-423d-871c-d3367a5ff66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:20.614469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23ee1c-6d64-423d-871c-d3367a5ff66e admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:20.615141 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23ee1c-6d64-423d-871c-d3367a5ff66e admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:20.615812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23ee1c-6d64-423d-871c-d3367a5ff66e admin admin] PATH_INFO: `/v3/projects/72892ffe8d164cba9654f22e072c5802/users/c4e52d5f5c094e8795564d81f0dc346b/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:20.662707 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c23ee1c-6d64-423d-871c-d3367a5ff66e admin admin] RBAC: Authorizing `identity:create_grant(project_id=72892ffe8d164cba9654f22e072c5802, user_id=c4e52d5f5c094e8795564d81f0dc346b, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:20.671818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c23ee1c-6d64-423d-871c-d3367a5ff66e admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:21.020724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3f166-401a-4b65-ab21-51e86c9817ad admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:21.022553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 897/1793] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:33:20 2022] PUT /identity/v3/projects/72892ffe8d164cba9654f22e072c5802/users/c4e52d5f5c094e8795564d81f0dc346b/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:21.047324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3f166-401a-4b65-ab21-51e86c9817ad admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:21.294405 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef844b-33a3-4688-8e2e-097434fc8581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:21.296710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef844b-33a3-4688-8e2e-097434fc8581 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:21.297662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef844b-33a3-4688-8e2e-097434fc8581 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:21.298603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef844b-33a3-4688-8e2e-097434fc8581 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:21.299966 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef844b-33a3-4688-8e2e-097434fc8581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:21.307296 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef844b-33a3-4688-8e2e-097434fc8581 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:21.353563 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 897/1794] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:33:21.355636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23ee1c-6d64-423d-871c-d3367a5ff66e admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:21.394225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23ee1c-6d64-423d-871c-d3367a5ff66e admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:21.422132 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87c5eb-cb89-4b33-ad6b-b7fdc2816420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:21.422964 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87c5eb-cb89-4b33-ad6b-b7fdc2816420 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:21.423467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87c5eb-cb89-4b33-ad6b-b7fdc2816420 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:21.423931 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87c5eb-cb89-4b33-ad6b-b7fdc2816420 admin admin] PATH_INFO: `/v3/projects/72892ffe8d164cba9654f22e072c5802/users/c4e52d5f5c094e8795564d81f0dc346b/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:21.436220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87c5eb-cb89-4b33-ad6b-b7fdc2816420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72892ffe8d164cba9654f22e072c5802, user_id=c4e52d5f5c094e8795564d81f0dc346b, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:21.453977 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87c5eb-cb89-4b33-ad6b-b7fdc2816420 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:21.577534 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 898/1795] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:33:21 2022] PUT /identity/v3/projects/72892ffe8d164cba9654f22e072c5802/users/c4e52d5f5c094e8795564d81f0dc346b/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:21.610446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68654a9b-bc5b-4202-986a-a94e467a9088 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:21.611208 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68654a9b-bc5b-4202-986a-a94e467a9088 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:21.611909 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68654a9b-bc5b-4202-986a-a94e467a9088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:21.790452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-68654a9b-bc5b-4202-986a-a94e467a9088 None None] MFA Rules not processed for user `c4e52d5f5c094e8795564d81f0dc346b`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:21.793965 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-68654a9b-bc5b-4202-986a-a94e467a9088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:22.346665 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87c5eb-cb89-4b33-ad6b-b7fdc2816420 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:22.347708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 898/1796] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 739 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:22.352095 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6e87c5eb-cb89-4b33-ad6b-b7fdc2816420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:22.643651 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87c5eb-cb89-4b33-ad6b-b7fdc2816420 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:22.674752 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80f112-d9ea-4745-b41f-23fab4f9f220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:22.677258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80f112-d9ea-4745-b41f-23fab4f9f220 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:22.677686 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80f112-d9ea-4745-b41f-23fab4f9f220 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:22.680419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80f112-d9ea-4745-b41f-23fab4f9f220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:22.706577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80f112-d9ea-4745-b41f-23fab4f9f220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:22.713792 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80f112-d9ea-4745-b41f-23fab4f9f220 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:22.744395 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 899/1797] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:22 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:33:23.138935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68654a9b-bc5b-4202-986a-a94e467a9088 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:23.177457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68654a9b-bc5b-4202-986a-a94e467a9088 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:23.208289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5cadc-7156-438f-9817-680342662d43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:23.209291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5cadc-7156-438f-9817-680342662d43 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:23.209762 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5cadc-7156-438f-9817-680342662d43 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:23.211426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5cadc-7156-438f-9817-680342662d43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:23.379312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:23.551337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5cadc-7156-438f-9817-680342662d43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:23.554157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5cadc-7156-438f-9817-680342662d43 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:23.581073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 899/1798] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:33:24.570093 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493e2cd-7f63-42b8-be15-f7b0ec83e21a None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:24.570093 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493e2cd-7f63-42b8-be15-f7b0ec83e21a None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:24.570093 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493e2cd-7f63-42b8-be15-f7b0ec83e21a None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:24.570093 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 900/1799] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 7 13:33:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:33:24.593791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f706051-9990-45d5-9a8f-43cb5a20634b None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:24.594730 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f706051-9990-45d5-9a8f-43cb5a20634b None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:24.594950 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f706051-9990-45d5-9a8f-43cb5a20634b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:24.847621 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-6f706051-9990-45d5-9a8f-43cb5a20634b None None] MFA Rules not processed for user `97c363046db444cbb3d4346066f9ad42`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:24.864305 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-6f706051-9990-45d5-9a8f-43cb5a20634b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:25.279980 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 900/1800] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 7 13:33:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 689 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:33:26.313245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4493e2cd-7f63-42b8-be15-f7b0ec83e21a None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:26.322283 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-4493e2cd-7f63-42b8-be15-f7b0ec83e21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:26.373855 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4493e2cd-7f63-42b8-be15-f7b0ec83e21a None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:26.384045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41af50-4355-4f39-b740-9928ac75a72d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:26.385074 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41af50-4355-4f39-b740-9928ac75a72d service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:26.385630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41af50-4355-4f39-b740-9928ac75a72d service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:26.386711 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41af50-4355-4f39-b740-9928ac75a72d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:26.571683 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41af50-4355-4f39-b740-9928ac75a72d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:26.571683 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41af50-4355-4f39-b740-9928ac75a72d service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:26.571683 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 901/1801] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:33:27.441035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931c387-882a-4811-a018-baa3c42ec51e None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:27.442161 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931c387-882a-4811-a018-baa3c42ec51e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:27.442601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931c387-882a-4811-a018-baa3c42ec51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:27.453848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf142b9-5fde-40fa-b823-beeb9d6a0b9e None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:27.460599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf142b9-5fde-40fa-b823-beeb9d6a0b9e None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:27.460599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf142b9-5fde-40fa-b823-beeb9d6a0b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:27.484971 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-7931c387-882a-4811-a018-baa3c42ec51e None None] MFA Rules not processed for user `32401931d75649018f6b0e55144bb5f2`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:27.490144 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7931c387-882a-4811-a018-baa3c42ec51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:27.516612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-dbf142b9-5fde-40fa-b823-beeb9d6a0b9e None None] MFA Rules not processed for user `7e5f1078006549c4b93a175f733b5b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:27.518259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-dbf142b9-5fde-40fa-b823-beeb9d6a0b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:27.596120 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:27.709496 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 902/1802] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:27.712691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 901/1803] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:27.733979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf142b9-5fde-40fa-b823-beeb9d6a0b9e None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:27.750961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf142b9-5fde-40fa-b823-beeb9d6a0b9e None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:27.755580 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd6b64-2e6d-46f2-a6cb-83b79cbaa301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:27.756364 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd6b64-2e6d-46f2-a6cb-83b79cbaa301 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:27.756760 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd6b64-2e6d-46f2-a6cb-83b79cbaa301 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:27.757145 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd6b64-2e6d-46f2-a6cb-83b79cbaa301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:27.759381 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-fccd6b64-2e6d-46f2-a6cb-83b79cbaa301 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:27.900927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd6b64-2e6d-46f2-a6cb-83b79cbaa301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:27.900927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd6b64-2e6d-46f2-a6cb-83b79cbaa301 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:27.927124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 903/1804] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:33:31.297518 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931c387-882a-4811-a018-baa3c42ec51e None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:31.313827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931c387-882a-4811-a018-baa3c42ec51e None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:31.322938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f361f-e7b3-428a-b4c0-72eb5a8af4f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:31.323653 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f361f-e7b3-428a-b4c0-72eb5a8af4f6 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:31.323929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f361f-e7b3-428a-b4c0-72eb5a8af4f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:31.324220 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f361f-e7b3-428a-b4c0-72eb5a8af4f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:31.341230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f361f-e7b3-428a-b4c0-72eb5a8af4f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:31.344365 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f361f-e7b3-428a-b4c0-72eb5a8af4f6 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:31.394945 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 902/1805] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:33:33.998245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd6b64-2e6d-46f2-a6cb-83b79cbaa301 service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:34.020648 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd6b64-2e6d-46f2-a6cb-83b79cbaa301 service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:34.040312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162f101-ce8b-4d44-85d0-9bee5b133ccd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:34.041339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162f101-ce8b-4d44-85d0-9bee5b133ccd service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:34.041751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162f101-ce8b-4d44-85d0-9bee5b133ccd service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:34.043376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162f101-ce8b-4d44-85d0-9bee5b133ccd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:34.050011 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-1162f101-ce8b-4d44-85d0-9bee5b133ccd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:34.263360 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162f101-ce8b-4d44-85d0-9bee5b133ccd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:34.266476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162f101-ce8b-4d44-85d0-9bee5b133ccd service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:34.288550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 904/1806] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:33:34.811761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1e478-82c1-4b4e-b468-a4600fca551b None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:34.812865 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1e478-82c1-4b4e-b468-a4600fca551b None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:34.813557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1e478-82c1-4b4e-b468-a4600fca551b None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:34.815709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 903/1807] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 7 13:33:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:33:34.831909 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c13b40-f284-4447-a902-1e625ff4dfdf None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:34.832396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c13b40-f284-4447-a902-1e625ff4dfdf None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:34.832886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c13b40-f284-4447-a902-1e625ff4dfdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:34.878433 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-00c13b40-f284-4447-a902-1e625ff4dfdf None None] MFA Rules not processed for user `97c363046db444cbb3d4346066f9ad42`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:34.880380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-00c13b40-f284-4447-a902-1e625ff4dfdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:34.893448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 905/1808] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 7 13:33:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:35.508676 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1e478-82c1-4b4e-b468-a4600fca551b None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:35.514470 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-71b1e478-82c1-4b4e-b468-a4600fca551b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:35.559803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1e478-82c1-4b4e-b468-a4600fca551b None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:35.569637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbba534-3517-4397-9403-595caadc1cc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:35.570477 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbba534-3517-4397-9403-595caadc1cc5 service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:35.571143 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbba534-3517-4397-9403-595caadc1cc5 service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:35.571866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbba534-3517-4397-9403-595caadc1cc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:35.590818 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbba534-3517-4397-9403-595caadc1cc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:35.595457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbba534-3517-4397-9403-595caadc1cc5 service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:35.623290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 904/1809] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:33:38.297277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c13b40-f284-4447-a902-1e625ff4dfdf None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:38.413469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:38.426017 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c13b40-f284-4447-a902-1e625ff4dfdf None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:38.434611 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30682635-405b-4944-8db6-1a4f502b033a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:38.435577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30682635-405b-4944-8db6-1a4f502b033a admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:38.435981 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30682635-405b-4944-8db6-1a4f502b033a admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:38.443214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30682635-405b-4944-8db6-1a4f502b033a admin admin] PATH_INFO: `/v3/users/4e82898a6f044abea6d099e784b5c494` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:38.443214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30682635-405b-4944-8db6-1a4f502b033a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e82898a6f044abea6d099e784b5c494)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:38.443214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30682635-405b-4944-8db6-1a4f502b033a admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:38.843351 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-30682635-405b-4944-8db6-1a4f502b033a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e82898a6f044abea6d099e784b5c494'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:38.868619 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-30682635-405b-4944-8db6-1a4f502b033a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e82898a6f044abea6d099e784b5c494'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:38.869433 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-30682635-405b-4944-8db6-1a4f502b033a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e82898a6f044abea6d099e784b5c494'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:38.893584 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-30682635-405b-4944-8db6-1a4f502b033a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e82898a6f044abea6d099e784b5c494'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:38.894746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-30682635-405b-4944-8db6-1a4f502b033a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e82898a6f044abea6d099e784b5c494'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:39.073521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 906/1810] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:33:38 2022] DELETE /identity/v3/users/4e82898a6f044abea6d099e784b5c494 => generated 0 bytes in 777 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:39.079650 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ef046-81aa-4ad6-8fd5-58f49f67c6a6 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:39.085361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ef046-81aa-4ad6-8fd5-58f49f67c6a6 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:39.085810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ef046-81aa-4ad6-8fd5-58f49f67c6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:39.155420 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-567ef046-81aa-4ad6-8fd5-58f49f67c6a6 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:39.157979 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-567ef046-81aa-4ad6-8fd5-58f49f67c6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:39.230351 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:39.306637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 905/1811] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:39.327165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30682635-405b-4944-8db6-1a4f502b033a admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:39.330840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-30682635-405b-4944-8db6-1a4f502b033a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:39.543731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30682635-405b-4944-8db6-1a4f502b033a admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:39.551502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4e794-81e2-435a-b591-35fc96214f86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:39.552691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4e794-81e2-435a-b591-35fc96214f86 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:39.553850 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4e794-81e2-435a-b591-35fc96214f86 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:39.554423 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4e794-81e2-435a-b591-35fc96214f86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:39.586663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4e794-81e2-435a-b591-35fc96214f86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:39.586663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f4e794-81e2-435a-b591-35fc96214f86 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:39.604922 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 907/1812] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:33:39.615018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dfd7b-fef6-4319-82a5-6770c44b6370 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:39.615305 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dfd7b-fef6-4319-82a5-6770c44b6370 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:39.615654 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dfd7b-fef6-4319-82a5-6770c44b6370 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:39.621938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 906/1813] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 7 13:33:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:33:39.663835 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e1521-2b67-48e5-bd93-1f0286081502 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:39.664769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e1521-2b67-48e5-bd93-1f0286081502 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:39.665873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e1521-2b67-48e5-bd93-1f0286081502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:39.707583 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-6c5e1521-2b67-48e5-bd93-1f0286081502 None None] MFA Rules not processed for user `97c363046db444cbb3d4346066f9ad42`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:39.709742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6c5e1521-2b67-48e5-bd93-1f0286081502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:39.844563 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 908/1814] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 7 13:33:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:39.921075 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8dfd7b-fef6-4319-82a5-6770c44b6370 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:39.922468 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0b8dfd7b-fef6-4319-82a5-6770c44b6370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:39.944874 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8dfd7b-fef6-4319-82a5-6770c44b6370 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:39.953045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f292e4-3569-443b-9190-666c47dca21c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:39.953664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f292e4-3569-443b-9190-666c47dca21c service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:39.953920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f292e4-3569-443b-9190-666c47dca21c service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:39.954217 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f292e4-3569-443b-9190-666c47dca21c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:39.966107 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f292e4-3569-443b-9190-666c47dca21c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:39.967908 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f292e4-3569-443b-9190-666c47dca21c service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:39.981791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 907/1815] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:33:40.621624 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e1521-2b67-48e5-bd93-1f0286081502 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:40.625377 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6c5e1521-2b67-48e5-bd93-1f0286081502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:40.672375 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e1521-2b67-48e5-bd93-1f0286081502 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:40.681689 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945eacd4-d71a-4a1f-be83-8bce39d149ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:40.682865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945eacd4-d71a-4a1f-be83-8bce39d149ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:40.683499 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945eacd4-d71a-4a1f-be83-8bce39d149ce admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:40.684524 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945eacd4-d71a-4a1f-be83-8bce39d149ce admin admin] PATH_INFO: `/v3/projects/f91810e2c6fb4ff3bd8b3db8a796e7ae` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:40.686741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945eacd4-d71a-4a1f-be83-8bce39d149ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=f91810e2c6fb4ff3bd8b3db8a796e7ae)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:40.693395 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945eacd4-d71a-4a1f-be83-8bce39d149ce admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:40.707621 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:40.977945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-945eacd4-d71a-4a1f-be83-8bce39d149ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f91810e2c6fb4ff3bd8b3db8a796e7ae', 'request_id': 'req-945eacd4-d71a-4a1f-be83-8bce39d149ce'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:40.990755 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-945eacd4-d71a-4a1f-be83-8bce39d149ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f91810e2c6fb4ff3bd8b3db8a796e7ae', 'request_id': 'req-945eacd4-d71a-4a1f-be83-8bce39d149ce'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:41.154736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-945eacd4-d71a-4a1f-be83-8bce39d149ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f91810e2c6fb4ff3bd8b3db8a796e7ae', 'request_id': 'req-945eacd4-d71a-4a1f-be83-8bce39d149ce'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:41.183101 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-945eacd4-d71a-4a1f-be83-8bce39d149ce admin admin] The token cache is being invalidate because project f91810e2c6fb4ff3bd8b3db8a796e7ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:33:41.183504 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-945eacd4-d71a-4a1f-be83-8bce39d149ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:41.184982 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-945eacd4-d71a-4a1f-be83-8bce39d149ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:41.187867 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 909/1816] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:33:40 2022] DELETE /identity/v3/projects/f91810e2c6fb4ff3bd8b3db8a796e7ae => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:41.195052 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f292e4-3569-443b-9190-666c47dca21c service nova] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:41.213027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f292e4-3569-443b-9190-666c47dca21c service nova] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:41.391665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936e00f-e255-4136-87d6-7a220cff62c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:41.394458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936e00f-e255-4136-87d6-7a220cff62c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:41.394458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936e00f-e255-4136-87d6-7a220cff62c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:41.394458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936e00f-e255-4136-87d6-7a220cff62c8 admin admin] PATH_INFO: `/v3/users/c4e52d5f5c094e8795564d81f0dc346b` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:41.398127 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936e00f-e255-4136-87d6-7a220cff62c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4e52d5f5c094e8795564d81f0dc346b)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:41.405053 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936e00f-e255-4136-87d6-7a220cff62c8 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:41.723153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-3936e00f-e255-4136-87d6-7a220cff62c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4e52d5f5c094e8795564d81f0dc346b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:41.752958 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945eacd4-d71a-4a1f-be83-8bce39d149ce admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:41.757455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-945eacd4-d71a-4a1f-be83-8bce39d149ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:41.787024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945eacd4-d71a-4a1f-be83-8bce39d149ce admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:41.800311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d3413-8e4c-4bd6-8412-c163ec091235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:41.802410 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d3413-8e4c-4bd6-8412-c163ec091235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:41.803387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d3413-8e4c-4bd6-8412-c163ec091235 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:41.804250 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d3413-8e4c-4bd6-8412-c163ec091235 admin admin] PATH_INFO: `/v3/users/0c0bf538904f4d39bed7f29ab29bed1e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:41.808609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d3413-8e4c-4bd6-8412-c163ec091235 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c0bf538904f4d39bed7f29ab29bed1e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:41.812530 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d3413-8e4c-4bd6-8412-c163ec091235 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:41.827830 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-3936e00f-e255-4136-87d6-7a220cff62c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4e52d5f5c094e8795564d81f0dc346b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:41.828703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-3936e00f-e255-4136-87d6-7a220cff62c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4e52d5f5c094e8795564d81f0dc346b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:41.829473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-3936e00f-e255-4136-87d6-7a220cff62c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4e52d5f5c094e8795564d81f0dc346b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:41.847215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-3936e00f-e255-4136-87d6-7a220cff62c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4e52d5f5c094e8795564d81f0dc346b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:41.881195 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 908/1817] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:33:41 2022] DELETE /identity/v3/users/c4e52d5f5c094e8795564d81f0dc346b => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:42.097450 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e88d3413-8e4c-4bd6-8412-c163ec091235 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c0bf538904f4d39bed7f29ab29bed1e'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:42.142478 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e88d3413-8e4c-4bd6-8412-c163ec091235 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c0bf538904f4d39bed7f29ab29bed1e'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:42.152455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e88d3413-8e4c-4bd6-8412-c163ec091235 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c0bf538904f4d39bed7f29ab29bed1e'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:42.171098 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e88d3413-8e4c-4bd6-8412-c163ec091235 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c0bf538904f4d39bed7f29ab29bed1e'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:42.172577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e88d3413-8e4c-4bd6-8412-c163ec091235 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c0bf538904f4d39bed7f29ab29bed1e'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:42.303310 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 910/1818] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:33:41 2022] DELETE /identity/v3/users/0c0bf538904f4d39bed7f29ab29bed1e => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:33:42.306435 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6455dd-5d81-46b7-afd3-f727302c69c0 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:42.307359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6455dd-5d81-46b7-afd3-f727302c69c0 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:42.307833 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6455dd-5d81-46b7-afd3-f727302c69c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:42.335567 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-1c6455dd-5d81-46b7-afd3-f727302c69c0 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:42.337401 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1c6455dd-5d81-46b7-afd3-f727302c69c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:42.405900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:42.448761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 909/1819] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:42.582492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d3413-8e4c-4bd6-8412-c163ec091235 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:42.584981 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e88d3413-8e4c-4bd6-8412-c163ec091235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:42.779379 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d3413-8e4c-4bd6-8412-c163ec091235 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:42.784970 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2cca8-a908-4987-8a45-707b4dadebae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:42.785764 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2cca8-a908-4987-8a45-707b4dadebae service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:42.786130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2cca8-a908-4987-8a45-707b4dadebae service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:42.786614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2cca8-a908-4987-8a45-707b4dadebae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:42.817486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2cca8-a908-4987-8a45-707b4dadebae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:42.822129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2cca8-a908-4987-8a45-707b4dadebae service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:42.842672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 911/1820] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:33:43.814905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6455dd-5d81-46b7-afd3-f727302c69c0 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:43.821378 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1c6455dd-5d81-46b7-afd3-f727302c69c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:43.843086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6455dd-5d81-46b7-afd3-f727302c69c0 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:43.859732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ccd91-b5ac-4401-a56b-0750a4cf4ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:43.861243 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ccd91-b5ac-4401-a56b-0750a4cf4ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:43.861796 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ccd91-b5ac-4401-a56b-0750a4cf4ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:43.862301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ccd91-b5ac-4401-a56b-0750a4cf4ed8 admin admin] PATH_INFO: `/v3/projects/72892ffe8d164cba9654f22e072c5802` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:43.864600 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ccd91-b5ac-4401-a56b-0750a4cf4ed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72892ffe8d164cba9654f22e072c5802)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:43.871084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ccd91-b5ac-4401-a56b-0750a4cf4ed8 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:44.134607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-097ccd91-b5ac-4401-a56b-0750a4cf4ed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72892ffe8d164cba9654f22e072c5802', 'request_id': 'req-097ccd91-b5ac-4401-a56b-0750a4cf4ed8'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:44.158900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-097ccd91-b5ac-4401-a56b-0750a4cf4ed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72892ffe8d164cba9654f22e072c5802', 'request_id': 'req-097ccd91-b5ac-4401-a56b-0750a4cf4ed8'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:44.162236 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2cca8-a908-4987-8a45-707b4dadebae service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:44.164664 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-bdb2cca8-a908-4987-8a45-707b4dadebae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:44.226185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2cca8-a908-4987-8a45-707b4dadebae service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:44.310738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-097ccd91-b5ac-4401-a56b-0750a4cf4ed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72892ffe8d164cba9654f22e072c5802', 'request_id': 'req-097ccd91-b5ac-4401-a56b-0750a4cf4ed8'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:44.347171 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-097ccd91-b5ac-4401-a56b-0750a4cf4ed8 admin admin] The token cache is being invalidate because project 72892ffe8d164cba9654f22e072c5802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:33:44.347171 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-097ccd91-b5ac-4401-a56b-0750a4cf4ed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:44.353105 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-097ccd91-b5ac-4401-a56b-0750a4cf4ed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:44.356326 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 910/1821] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:33:43 2022] DELETE /identity/v3/projects/72892ffe8d164cba9654f22e072c5802 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:44.363960 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6d7f4-58f4-4223-b57c-e65cdf58a1eb None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:44.364604 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6d7f4-58f4-4223-b57c-e65cdf58a1eb None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:44.365274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6d7f4-58f4-4223-b57c-e65cdf58a1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:44.398519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:44.399866 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:44.400428 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:44.401550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2 admin admin] PATH_INFO: `/v3/projects/7be5c62f81284249ab09bf6c72fc65ca` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:44.403964 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7be5c62f81284249ab09bf6c72fc65ca)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:44.407902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:44.415920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-6eb6d7f4-58f4-4223-b57c-e65cdf58a1eb None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:44.417942 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-6eb6d7f4-58f4-4223-b57c-e65cdf58a1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:44.430923 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 911/1822] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:33:44.441775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b61a20-5c1d-482e-a920-bd6be8200cfb None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:44.442422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b61a20-5c1d-482e-a920-bd6be8200cfb None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:44.442891 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b61a20-5c1d-482e-a920-bd6be8200cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:44.471877 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-22b61a20-5c1d-482e-a920-bd6be8200cfb None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:44.473554 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-22b61a20-5c1d-482e-a920-bd6be8200cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:44.498597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 912/1823] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:44.499819 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b61a20-5c1d-482e-a920-bd6be8200cfb None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:44.517923 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b61a20-5c1d-482e-a920-bd6be8200cfb None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:44.530389 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201777ff-460a-45f6-9378-c2a6d1017049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:44.535069 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201777ff-460a-45f6-9378-c2a6d1017049 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:44.535069 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201777ff-460a-45f6-9378-c2a6d1017049 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:44.535069 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201777ff-460a-45f6-9378-c2a6d1017049 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:44.535815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201777ff-460a-45f6-9378-c2a6d1017049 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:33:44.537679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201777ff-460a-45f6-9378-c2a6d1017049 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:44.543202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201777ff-460a-45f6-9378-c2a6d1017049 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:44.614449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 913/1824] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:33:44.618900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201777ff-460a-45f6-9378-c2a6d1017049 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:44.676035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201777ff-460a-45f6-9378-c2a6d1017049 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:44.696366 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7be5c62f81284249ab09bf6c72fc65ca', 'request_id': 'req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:44.720567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7be5c62f81284249ab09bf6c72fc65ca', 'request_id': 'req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:44.854758 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7be5c62f81284249ab09bf6c72fc65ca', 'request_id': 'req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:44.897110 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2 admin admin] The token cache is being invalidate because project 7be5c62f81284249ab09bf6c72fc65ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:33:44.902074 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:44.904951 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:44.908825 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4581c6-0b5c-4704-becf-87b3c5aa2ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:44.910021 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4581c6-0b5c-4704-becf-87b3c5aa2ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:44.910553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4581c6-0b5c-4704-becf-87b3c5aa2ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:44.911221 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4581c6-0b5c-4704-becf-87b3c5aa2ff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:44.912959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 912/1825] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:33:44 2022] DELETE /identity/v3/projects/7be5c62f81284249ab09bf6c72fc65ca => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:44.914334 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4581c6-0b5c-4704-becf-87b3c5aa2ff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:44.917734 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:44.919449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4581c6-0b5c-4704-becf-87b3c5aa2ff4 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:44.925897 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:44.990056 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c36b2e-adc1-4403-aeb6-bdfe7e1652f2 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:45.002484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7deeb-b4ca-4fbd-b549-980c19e5c227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:45.003641 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7deeb-b4ca-4fbd-b549-980c19e5c227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:45.003641 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7deeb-b4ca-4fbd-b549-980c19e5c227 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:45.003942 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7deeb-b4ca-4fbd-b549-980c19e5c227 admin admin] PATH_INFO: `/v3/users/286cbafca3804eb38646677b739c486f` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:45.005145 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd7deeb-b4ca-4fbd-b549-980c19e5c227 admin admin] RBAC: Authorizing `identity:delete_user(user_id=286cbafca3804eb38646677b739c486f)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:45.007891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd7deeb-b4ca-4fbd-b549-980c19e5c227 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:45.142733 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 914/1826] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:33:44 2022] POST /identity/v3/projects => generated 399 bytes in 525 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:33:45.144865 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4581c6-0b5c-4704-becf-87b3c5aa2ff4 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:45.153878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-4e4581c6-0b5c-4704-becf-87b3c5aa2ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:45.320117 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-dcd7deeb-b4ca-4fbd-b549-980c19e5c227 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '286cbafca3804eb38646677b739c486f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:45.348140 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-dcd7deeb-b4ca-4fbd-b549-980c19e5c227 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '286cbafca3804eb38646677b739c486f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:45.349421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-dcd7deeb-b4ca-4fbd-b549-980c19e5c227 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '286cbafca3804eb38646677b739c486f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:45.359672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-dcd7deeb-b4ca-4fbd-b549-980c19e5c227 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '286cbafca3804eb38646677b739c486f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:45.360773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-dcd7deeb-b4ca-4fbd-b549-980c19e5c227 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '286cbafca3804eb38646677b739c486f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:45.369800 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4581c6-0b5c-4704-becf-87b3c5aa2ff4 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:45.379939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd49ac-d940-497f-94a5-3fbabc1dfb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:45.380455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd49ac-d940-497f-94a5-3fbabc1dfb77 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:45.381054 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd49ac-d940-497f-94a5-3fbabc1dfb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:45.381648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd49ac-d940-497f-94a5-3fbabc1dfb77 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:45.383068 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd49ac-d940-497f-94a5-3fbabc1dfb77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:45.386884 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd49ac-d940-497f-94a5-3fbabc1dfb77 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:45.525118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 913/1827] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:33:44 2022] DELETE /identity/v3/users/286cbafca3804eb38646677b739c486f => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:45.532434 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 915/1828] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:33:45 2022] POST /identity/v3/users => generated 300 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:33:45.538266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7deeb-b4ca-4fbd-b549-980c19e5c227 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:45.548467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-dcd7deeb-b4ca-4fbd-b549-980c19e5c227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:45.734520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7deeb-b4ca-4fbd-b549-980c19e5c227 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:45.740649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34449cc-4d91-4c43-881b-b8bd096b7875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:45.741435 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34449cc-4d91-4c43-881b-b8bd096b7875 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:45.741763 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34449cc-4d91-4c43-881b-b8bd096b7875 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:45.742406 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34449cc-4d91-4c43-881b-b8bd096b7875 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:45.743283 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34449cc-4d91-4c43-881b-b8bd096b7875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:45.746438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34449cc-4d91-4c43-881b-b8bd096b7875 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:45.771721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 914/1829] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:33:45.773065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd49ac-d940-497f-94a5-3fbabc1dfb77 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:45.788821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd49ac-d940-497f-94a5-3fbabc1dfb77 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:45.795664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990127c-5f76-4d72-a787-0bee097a24ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:45.796590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990127c-5f76-4d72-a787-0bee097a24ac admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:45.796954 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990127c-5f76-4d72-a787-0bee097a24ac admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:45.797368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990127c-5f76-4d72-a787-0bee097a24ac admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:45.798409 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990127c-5f76-4d72-a787-0bee097a24ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:45.800831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990127c-5f76-4d72-a787-0bee097a24ac admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:45.819278 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 916/1830] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:33:45.822115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34449cc-4d91-4c43-881b-b8bd096b7875 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:45.837575 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34449cc-4d91-4c43-881b-b8bd096b7875 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:45.844042 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f43fe-6869-4f53-8c2b-d1289985c189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:45.844788 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f43fe-6869-4f53-8c2b-d1289985c189 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:45.845111 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f43fe-6869-4f53-8c2b-d1289985c189 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:45.845367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f43fe-6869-4f53-8c2b-d1289985c189 admin admin] PATH_INFO: `/v3/projects/bf386efa76c44687a401e3559f157fc5/users/c9c6ef63d1f041f6a7d320e47c864c0f/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:45.930234 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f43fe-6869-4f53-8c2b-d1289985c189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf386efa76c44687a401e3559f157fc5, user_id=c9c6ef63d1f041f6a7d320e47c864c0f, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:45.932783 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f43fe-6869-4f53-8c2b-d1289985c189 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:45.985825 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:46.080522 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 915/1831] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:33:45 2022] PUT /identity/v3/projects/bf386efa76c44687a401e3559f157fc5/users/c9c6ef63d1f041f6a7d320e47c864c0f/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:46.098682 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6dfe5-e5ce-446c-92ae-99eeb59fc0e3 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:46.099593 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6dfe5-e5ce-446c-92ae-99eeb59fc0e3 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:46.099593 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6dfe5-e5ce-446c-92ae-99eeb59fc0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:46.153280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-0ee6dfe5-e5ce-446c-92ae-99eeb59fc0e3 None None] MFA Rules not processed for user `c9c6ef63d1f041f6a7d320e47c864c0f`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:46.155406 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0ee6dfe5-e5ce-446c-92ae-99eeb59fc0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:46.316153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 917/1832] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:46 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:33:46.323875 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f43fe-6869-4f53-8c2b-d1289985c189 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:46.458333 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f43fe-6869-4f53-8c2b-d1289985c189 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:46.470492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fb071-e3d0-4203-b08a-b4ec9f1a4354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:46.471877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fb071-e3d0-4203-b08a-b4ec9f1a4354 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:46.474804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fb071-e3d0-4203-b08a-b4ec9f1a4354 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:46.476287 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fb071-e3d0-4203-b08a-b4ec9f1a4354 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:46.478734 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4fb071-e3d0-4203-b08a-b4ec9f1a4354 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:46.483891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4fb071-e3d0-4203-b08a-b4ec9f1a4354 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:46.727015 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6dfe5-e5ce-446c-92ae-99eeb59fc0e3 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:46.739035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 916/1833] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:33:46 2022] POST /identity/v3/projects => generated 397 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:33:46.751011 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6dfe5-e5ce-446c-92ae-99eeb59fc0e3 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:46.884017 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1de3b8-f2bb-49dd-b557-e33b31dc4be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:46.888647 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1de3b8-f2bb-49dd-b557-e33b31dc4be4 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:46.889324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1de3b8-f2bb-49dd-b557-e33b31dc4be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:46.891880 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1de3b8-f2bb-49dd-b557-e33b31dc4be4 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:46.893259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1de3b8-f2bb-49dd-b557-e33b31dc4be4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:46.897924 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1de3b8-f2bb-49dd-b557-e33b31dc4be4 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:47.124227 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 918/1834] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:33:46 2022] POST /identity/v3/users => generated 305 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:33:47.134227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fb071-e3d0-4203-b08a-b4ec9f1a4354 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:47.156670 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fb071-e3d0-4203-b08a-b4ec9f1a4354 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:47.172176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15ae9c-9f6d-4511-bf62-78c10e0481f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:47.173678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15ae9c-9f6d-4511-bf62-78c10e0481f5 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:47.174393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15ae9c-9f6d-4511-bf62-78c10e0481f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:47.175313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15ae9c-9f6d-4511-bf62-78c10e0481f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:47.176415 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15ae9c-9f6d-4511-bf62-78c10e0481f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:47.179743 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15ae9c-9f6d-4511-bf62-78c10e0481f5 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:47.206566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 917/1835] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:33:47.213318 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1de3b8-f2bb-49dd-b557-e33b31dc4be4 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:47.234356 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1de3b8-f2bb-49dd-b557-e33b31dc4be4 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:47.245563 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c83895-191f-46f2-acb3-c7a1ffe62532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:47.246897 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c83895-191f-46f2-acb3-c7a1ffe62532 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:47.259341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c83895-191f-46f2-acb3-c7a1ffe62532 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:47.259341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c83895-191f-46f2-acb3-c7a1ffe62532 admin admin] PATH_INFO: `/v3/projects/0961542fe2624e599ee3fd7d4fb52f5b/users/3f998265a2e1458f9d40fd519eb38e9e/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:47.288533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c83895-191f-46f2-acb3-c7a1ffe62532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0961542fe2624e599ee3fd7d4fb52f5b, user_id=3f998265a2e1458f9d40fd519eb38e9e, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:47.298243 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c83895-191f-46f2-acb3-c7a1ffe62532 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:47.512716 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 919/1836] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:33:47 2022] PUT /identity/v3/projects/0961542fe2624e599ee3fd7d4fb52f5b/users/3f998265a2e1458f9d40fd519eb38e9e/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:33:47.515113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15ae9c-9f6d-4511-bf62-78c10e0481f5 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:47.518744 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-cb15ae9c-9f6d-4511-bf62-78c10e0481f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:47.519968 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c83895-191f-46f2-acb3-c7a1ffe62532 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:47.566349 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15ae9c-9f6d-4511-bf62-78c10e0481f5 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:47.707879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c83895-191f-46f2-acb3-c7a1ffe62532 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:47.727843 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43213738-ecf7-4a10-ac4a-97ff3cda0010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:47.728964 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43213738-ecf7-4a10-ac4a-97ff3cda0010 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:47.729473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43213738-ecf7-4a10-ac4a-97ff3cda0010 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:47.730022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43213738-ecf7-4a10-ac4a-97ff3cda0010 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:47.731474 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43213738-ecf7-4a10-ac4a-97ff3cda0010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:47.735931 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43213738-ecf7-4a10-ac4a-97ff3cda0010 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:47.749330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9496284-47ae-4146-9c09-fc6754c29083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:47.750488 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9496284-47ae-4146-9c09-fc6754c29083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:47.750945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9496284-47ae-4146-9c09-fc6754c29083 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:47.751491 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9496284-47ae-4146-9c09-fc6754c29083 admin admin] PATH_INFO: `/v3/projects/3f339dfb911c441d80aab6fe7ba62095` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:47.753765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9496284-47ae-4146-9c09-fc6754c29083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f339dfb911c441d80aab6fe7ba62095)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:47.757677 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9496284-47ae-4146-9c09-fc6754c29083 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:47.759786 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 920/1837] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:33:47.766612 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43213738-ecf7-4a10-ac4a-97ff3cda0010 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:47.806430 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43213738-ecf7-4a10-ac4a-97ff3cda0010 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:47.816621 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7344b66-cfd1-4ed8-bb58-73d4ac1685cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:47.817979 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7344b66-cfd1-4ed8-bb58-73d4ac1685cc admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:47.818459 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7344b66-cfd1-4ed8-bb58-73d4ac1685cc admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:47.818930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7344b66-cfd1-4ed8-bb58-73d4ac1685cc admin admin] PATH_INFO: `/v3/projects/0961542fe2624e599ee3fd7d4fb52f5b/users/3f998265a2e1458f9d40fd519eb38e9e/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:47.822508 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7344b66-cfd1-4ed8-bb58-73d4ac1685cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0961542fe2624e599ee3fd7d4fb52f5b, user_id=3f998265a2e1458f9d40fd519eb38e9e, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:47.828458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7344b66-cfd1-4ed8-bb58-73d4ac1685cc admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:47.849375 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 921/1838] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:33:47 2022] PUT /identity/v3/projects/0961542fe2624e599ee3fd7d4fb52f5b/users/3f998265a2e1458f9d40fd519eb38e9e/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:33:47.869580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea1ab8-f45a-4ca1-ad3e-5f7e25e60baf None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:47.871150 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea1ab8-f45a-4ca1-ad3e-5f7e25e60baf None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:47.871434 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea1ab8-f45a-4ca1-ad3e-5f7e25e60baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:47.933837 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-1dea1ab8-f45a-4ca1-ad3e-5f7e25e60baf None None] MFA Rules not processed for user `3f998265a2e1458f9d40fd519eb38e9e`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:47.936581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1dea1ab8-f45a-4ca1-ad3e-5f7e25e60baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:48.000456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e9496284-47ae-4146-9c09-fc6754c29083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f339dfb911c441d80aab6fe7ba62095', 'request_id': 'req-e9496284-47ae-4146-9c09-fc6754c29083'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:48.020214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e9496284-47ae-4146-9c09-fc6754c29083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f339dfb911c441d80aab6fe7ba62095', 'request_id': 'req-e9496284-47ae-4146-9c09-fc6754c29083'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:48.124486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e9496284-47ae-4146-9c09-fc6754c29083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f339dfb911c441d80aab6fe7ba62095', 'request_id': 'req-e9496284-47ae-4146-9c09-fc6754c29083'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:48.151919 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 922/1839] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:47 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:48.161904 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e9496284-47ae-4146-9c09-fc6754c29083 admin admin] The token cache is being invalidate because project 3f339dfb911c441d80aab6fe7ba62095 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:33:48.162383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e9496284-47ae-4146-9c09-fc6754c29083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:48.164815 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e9496284-47ae-4146-9c09-fc6754c29083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:48.172460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 918/1840] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:33:47 2022] DELETE /identity/v3/projects/3f339dfb911c441d80aab6fe7ba62095 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:48.176171 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea1ab8-f45a-4ca1-ad3e-5f7e25e60baf None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:48.195137 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9496284-47ae-4146-9c09-fc6754c29083 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:48.195137 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e9496284-47ae-4146-9c09-fc6754c29083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:48.349609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9496284-47ae-4146-9c09-fc6754c29083 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:48.355261 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bd8d0-0152-4c97-a0d3-8f5a382a713d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:48.356139 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bd8d0-0152-4c97-a0d3-8f5a382a713d service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:48.356517 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bd8d0-0152-4c97-a0d3-8f5a382a713d service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:48.377652 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bd8d0-0152-4c97-a0d3-8f5a382a713d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:48.377652 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-1b3bd8d0-0152-4c97-a0d3-8f5a382a713d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:48.397788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea1ab8-f45a-4ca1-ad3e-5f7e25e60baf None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:48.529169 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162b5a4-94b8-44ad-8333-b5ad1b959756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:48.529967 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162b5a4-94b8-44ad-8333-b5ad1b959756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:48.530655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162b5a4-94b8-44ad-8333-b5ad1b959756 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:48.531021 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162b5a4-94b8-44ad-8333-b5ad1b959756 admin admin] PATH_INFO: `/v3/users/1d9551e37ad24a20bb1d6723bf512124` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:48.532461 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162b5a4-94b8-44ad-8333-b5ad1b959756 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d9551e37ad24a20bb1d6723bf512124)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:48.561002 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162b5a4-94b8-44ad-8333-b5ad1b959756 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:48.561971 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3bd8d0-0152-4c97-a0d3-8f5a382a713d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:48.561971 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3bd8d0-0152-4c97-a0d3-8f5a382a713d service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:48.572067 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 919/1841] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:48 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:33:48.769798 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:48.907983 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-d162b5a4-94b8-44ad-8333-b5ad1b959756 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d9551e37ad24a20bb1d6723bf512124'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:49.036860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bd8d0-0152-4c97-a0d3-8f5a382a713d service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:49.041676 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-d162b5a4-94b8-44ad-8333-b5ad1b959756 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d9551e37ad24a20bb1d6723bf512124'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:49.042758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-d162b5a4-94b8-44ad-8333-b5ad1b959756 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d9551e37ad24a20bb1d6723bf512124'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:49.043720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-d162b5a4-94b8-44ad-8333-b5ad1b959756 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d9551e37ad24a20bb1d6723bf512124'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:49.064618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bd8d0-0152-4c97-a0d3-8f5a382a713d service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:49.074520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09e115-e140-4afd-b08e-e4b33e29b3c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:49.075957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09e115-e140-4afd-b08e-e4b33e29b3c1 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:49.076857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09e115-e140-4afd-b08e-e4b33e29b3c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:49.077338 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-d162b5a4-94b8-44ad-8333-b5ad1b959756 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d9551e37ad24a20bb1d6723bf512124'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:49.077959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09e115-e140-4afd-b08e-e4b33e29b3c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:49.092878 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-1f09e115-e140-4afd-b08e-e4b33e29b3c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:49.139327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 923/1842] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:33:48 2022] DELETE /identity/v3/users/1d9551e37ad24a20bb1d6723bf512124 => generated 0 bytes in 966 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:49.299829 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09e115-e140-4afd-b08e-e4b33e29b3c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:49.303380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09e115-e140-4afd-b08e-e4b33e29b3c1 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:49.340568 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 920/1843] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:49 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:33:50.181198 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162b5a4-94b8-44ad-8333-b5ad1b959756 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:50.192955 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d162b5a4-94b8-44ad-8333-b5ad1b959756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:50.452644 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162b5a4-94b8-44ad-8333-b5ad1b959756 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:50.466887 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdabf910-5d5f-4414-b0b4-0b95e6282683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:50.467978 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabf910-5d5f-4414-b0b4-0b95e6282683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:50.468451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabf910-5d5f-4414-b0b4-0b95e6282683 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:50.468920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabf910-5d5f-4414-b0b4-0b95e6282683 admin admin] PATH_INFO: `/v3/projects/5afc0be3c2174c7d95c146a414a06696` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:50.471916 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdabf910-5d5f-4414-b0b4-0b95e6282683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5afc0be3c2174c7d95c146a414a06696)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:50.510205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdabf910-5d5f-4414-b0b4-0b95e6282683 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:50.748745 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-bdabf910-5d5f-4414-b0b4-0b95e6282683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5afc0be3c2174c7d95c146a414a06696', 'request_id': 'req-bdabf910-5d5f-4414-b0b4-0b95e6282683'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:50.777541 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-bdabf910-5d5f-4414-b0b4-0b95e6282683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5afc0be3c2174c7d95c146a414a06696', 'request_id': 'req-bdabf910-5d5f-4414-b0b4-0b95e6282683'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:50.938198 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-bdabf910-5d5f-4414-b0b4-0b95e6282683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5afc0be3c2174c7d95c146a414a06696', 'request_id': 'req-bdabf910-5d5f-4414-b0b4-0b95e6282683'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:50.960549 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-bdabf910-5d5f-4414-b0b4-0b95e6282683 admin admin] The token cache is being invalidate because project 5afc0be3c2174c7d95c146a414a06696 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:33:50.961078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-bdabf910-5d5f-4414-b0b4-0b95e6282683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:50.962330 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-bdabf910-5d5f-4414-b0b4-0b95e6282683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:50.966325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 924/1844] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:33:50 2022] DELETE /identity/v3/projects/5afc0be3c2174c7d95c146a414a06696 => generated 0 bytes in 786 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:50.974099 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594091ac-0f6d-46a2-af5a-9105c60cd43f None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:50.977706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594091ac-0f6d-46a2-af5a-9105c60cd43f None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:50.977706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594091ac-0f6d-46a2-af5a-9105c60cd43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:51.109058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-594091ac-0f6d-46a2-af5a-9105c60cd43f None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:51.115338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-594091ac-0f6d-46a2-af5a-9105c60cd43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:51.296811 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 921/1845] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:51.314302 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723e736-7f4a-4a79-8848-9a4413b6f778 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:51.314931 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723e736-7f4a-4a79-8848-9a4413b6f778 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:51.315573 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723e736-7f4a-4a79-8848-9a4413b6f778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:51.368268 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-5723e736-7f4a-4a79-8848-9a4413b6f778 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:51.370790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-5723e736-7f4a-4a79-8848-9a4413b6f778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:51.383062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 925/1846] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:51.393967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594091ac-0f6d-46a2-af5a-9105c60cd43f None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:51.412195 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594091ac-0f6d-46a2-af5a-9105c60cd43f None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:51.422243 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52ed53-a91b-439e-a33c-2302d14a5e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:51.422845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52ed53-a91b-439e-a33c-2302d14a5e67 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:51.430791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52ed53-a91b-439e-a33c-2302d14a5e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:51.430791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52ed53-a91b-439e-a33c-2302d14a5e67 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:51.430791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52ed53-a91b-439e-a33c-2302d14a5e67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:33:51.430791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52ed53-a91b-439e-a33c-2302d14a5e67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:51.430791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52ed53-a91b-439e-a33c-2302d14a5e67 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:51.452286 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 922/1847] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:33:51.459314 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723e736-7f4a-4a79-8848-9a4413b6f778 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:51.475287 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723e736-7f4a-4a79-8848-9a4413b6f778 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:51.491530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f21f0-93b5-42b6-8a9a-d1e5044df7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:51.492929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f21f0-93b5-42b6-8a9a-d1e5044df7bb admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:51.493706 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f21f0-93b5-42b6-8a9a-d1e5044df7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:51.497276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f21f0-93b5-42b6-8a9a-d1e5044df7bb admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:51.497620 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f21f0-93b5-42b6-8a9a-d1e5044df7bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:51.502691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f21f0-93b5-42b6-8a9a-d1e5044df7bb admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:51.712261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 926/1848] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:33:51 2022] POST /identity/v3/projects => generated 387 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:33:51.720691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52ed53-a91b-439e-a33c-2302d14a5e67 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:51.739811 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52ed53-a91b-439e-a33c-2302d14a5e67 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:51.836805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d006b-8552-42cb-b927-2b11d404841a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:51.837449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d006b-8552-42cb-b927-2b11d404841a admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:51.837732 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d006b-8552-42cb-b927-2b11d404841a admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:51.838057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d006b-8552-42cb-b927-2b11d404841a admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:51.838736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d006b-8552-42cb-b927-2b11d404841a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:51.841194 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d006b-8552-42cb-b927-2b11d404841a admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:52.078204 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 923/1849] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:33:51 2022] POST /identity/v3/users => generated 294 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:33:52.085785 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f21f0-93b5-42b6-8a9a-d1e5044df7bb admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:52.105978 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f21f0-93b5-42b6-8a9a-d1e5044df7bb admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:52.124438 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2b473-5761-4efa-a3ac-649c88ff6af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:52.125478 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2b473-5761-4efa-a3ac-649c88ff6af1 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:52.129549 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2b473-5761-4efa-a3ac-649c88ff6af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:52.129986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2b473-5761-4efa-a3ac-649c88ff6af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:52.131179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2b473-5761-4efa-a3ac-649c88ff6af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:52.133638 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2b473-5761-4efa-a3ac-649c88ff6af1 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:52.180622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d006b-8552-42cb-b927-2b11d404841a admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:52.180896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 927/1850] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:33:52.184434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d006b-8552-42cb-b927-2b11d404841a admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:52.193331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b568567-b6b4-4406-8880-860c78700108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:52.195236 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b568567-b6b4-4406-8880-860c78700108 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:52.195732 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b568567-b6b4-4406-8880-860c78700108 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:52.196179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b568567-b6b4-4406-8880-860c78700108 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:52.198267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b568567-b6b4-4406-8880-860c78700108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:52.202274 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b568567-b6b4-4406-8880-860c78700108 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:52.225460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 924/1851] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:33:52.232369 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2b473-5761-4efa-a3ac-649c88ff6af1 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:52.277746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2b473-5761-4efa-a3ac-649c88ff6af1 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:52.286926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359723c7-8254-4c21-b35d-625d1f24e46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:52.288452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359723c7-8254-4c21-b35d-625d1f24e46e admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:52.288882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359723c7-8254-4c21-b35d-625d1f24e46e admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:52.289499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359723c7-8254-4c21-b35d-625d1f24e46e admin admin] PATH_INFO: `/v3/projects/2eb504b6396f49eeb69d34a491f9f93a/users/7469a8ed97604d45830fdbc5c3e5ff26/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:52.331823 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359723c7-8254-4c21-b35d-625d1f24e46e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eb504b6396f49eeb69d34a491f9f93a, user_id=7469a8ed97604d45830fdbc5c3e5ff26, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:52.335730 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:52.340244 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359723c7-8254-4c21-b35d-625d1f24e46e admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:52.527173 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 928/1852] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:33:52 2022] PUT /identity/v3/projects/2eb504b6396f49eeb69d34a491f9f93a/users/7469a8ed97604d45830fdbc5c3e5ff26/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:33:52.545267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a98dfe-d556-4803-9e1b-b2a48239334c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:52.545583 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a98dfe-d556-4803-9e1b-b2a48239334c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:52.545867 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a98dfe-d556-4803-9e1b-b2a48239334c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:52.603236 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-78a98dfe-d556-4803-9e1b-b2a48239334c None None] MFA Rules not processed for user `7469a8ed97604d45830fdbc5c3e5ff26`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:52.607315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-78a98dfe-d556-4803-9e1b-b2a48239334c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:52.765735 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 925/1853] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:33:52.771788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359723c7-8254-4c21-b35d-625d1f24e46e admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:52.865857 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359723c7-8254-4c21-b35d-625d1f24e46e admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:52.875133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c940330-d36d-49f6-b295-b3d003175ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:52.876243 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c940330-d36d-49f6-b295-b3d003175ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:52.876749 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c940330-d36d-49f6-b295-b3d003175ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:52.877353 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c940330-d36d-49f6-b295-b3d003175ef9 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:52.878723 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c940330-d36d-49f6-b295-b3d003175ef9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:52.882661 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c940330-d36d-49f6-b295-b3d003175ef9 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:53.089350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 929/1854] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:33:52 2022] POST /identity/v3/projects => generated 389 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:33:53.092541 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a98dfe-d556-4803-9e1b-b2a48239334c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:53.106903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a98dfe-d556-4803-9e1b-b2a48239334c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:53.212481 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1f92c-fffd-4341-b649-fe60ea0ecb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:53.214416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1f92c-fffd-4341-b649-fe60ea0ecb6e admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:53.214998 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1f92c-fffd-4341-b649-fe60ea0ecb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:53.215535 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1f92c-fffd-4341-b649-fe60ea0ecb6e admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:53.216475 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1f92c-fffd-4341-b649-fe60ea0ecb6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:53.219130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1f92c-fffd-4341-b649-fe60ea0ecb6e admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:53.385067 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c940330-d36d-49f6-b295-b3d003175ef9 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:53.396675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 926/1855] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:33:53 2022] POST /identity/v3/users => generated 301 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:33:53.403015 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c940330-d36d-49f6-b295-b3d003175ef9 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:53.409737 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45dbd0d-0066-4b68-a2c9-04a0cf5bd4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:53.410786 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45dbd0d-0066-4b68-a2c9-04a0cf5bd4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:53.411300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45dbd0d-0066-4b68-a2c9-04a0cf5bd4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:53.411812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45dbd0d-0066-4b68-a2c9-04a0cf5bd4a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:53.412789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45dbd0d-0066-4b68-a2c9-04a0cf5bd4a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:53.414912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45dbd0d-0066-4b68-a2c9-04a0cf5bd4a8 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:53.436684 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 930/1856] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:33:53.448386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1f92c-fffd-4341-b649-fe60ea0ecb6e admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:53.484419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1f92c-fffd-4341-b649-fe60ea0ecb6e admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:53.504615 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3ca96-9872-4835-adef-11069ec39aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:53.505506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3ca96-9872-4835-adef-11069ec39aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:53.505858 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3ca96-9872-4835-adef-11069ec39aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:53.506213 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3ca96-9872-4835-adef-11069ec39aa0 admin admin] PATH_INFO: `/v3/projects/5419f47d6d024e4d8771f6e99bb5f14c/users/b0616d436e204326b706054adecd180c/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:53.545854 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3ca96-9872-4835-adef-11069ec39aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5419f47d6d024e4d8771f6e99bb5f14c, user_id=b0616d436e204326b706054adecd180c, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:53.552708 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3ca96-9872-4835-adef-11069ec39aa0 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:53.673017 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 927/1857] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:33:53 2022] PUT /identity/v3/projects/5419f47d6d024e4d8771f6e99bb5f14c/users/b0616d436e204326b706054adecd180c/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:53.680007 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45dbd0d-0066-4b68-a2c9-04a0cf5bd4a8 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:53.707604 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45dbd0d-0066-4b68-a2c9-04a0cf5bd4a8 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:53.892229 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f5d93-50a9-4411-a449-18ef04c00c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:53.902119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f5d93-50a9-4411-a449-18ef04c00c89 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:53.903722 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f5d93-50a9-4411-a449-18ef04c00c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:53.904881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f5d93-50a9-4411-a449-18ef04c00c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:53.908048 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f5d93-50a9-4411-a449-18ef04c00c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:53.911371 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f5d93-50a9-4411-a449-18ef04c00c89 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:53.940647 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 931/1858] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:33:53.949233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3ca96-9872-4835-adef-11069ec39aa0 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:53.996696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3ca96-9872-4835-adef-11069ec39aa0 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:54.013299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa15f0-1db2-4753-88ef-7d828eba67da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:54.014492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa15f0-1db2-4753-88ef-7d828eba67da admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:54.014884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa15f0-1db2-4753-88ef-7d828eba67da admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:54.015299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa15f0-1db2-4753-88ef-7d828eba67da admin admin] PATH_INFO: `/v3/projects/5419f47d6d024e4d8771f6e99bb5f14c/users/b0616d436e204326b706054adecd180c/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:54.024715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa15f0-1db2-4753-88ef-7d828eba67da admin admin] RBAC: Authorizing `identity:create_grant(project_id=5419f47d6d024e4d8771f6e99bb5f14c, user_id=b0616d436e204326b706054adecd180c, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:54.029310 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa15f0-1db2-4753-88ef-7d828eba67da admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:54.073964 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 928/1859] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:33:53 2022] PUT /identity/v3/projects/5419f47d6d024e4d8771f6e99bb5f14c/users/b0616d436e204326b706054adecd180c/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:54.094301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00d338-e8d2-453f-a2ee-95cf122ec03c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:54.094776 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00d338-e8d2-453f-a2ee-95cf122ec03c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:54.100304 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00d338-e8d2-453f-a2ee-95cf122ec03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:54.173198 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-8b00d338-e8d2-453f-a2ee-95cf122ec03c None None] MFA Rules not processed for user `b0616d436e204326b706054adecd180c`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:54.179004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:54.180879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8b00d338-e8d2-453f-a2ee-95cf122ec03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:54.380880 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 932/1860] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:54 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:54.415705 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa15f0-1db2-4753-88ef-7d828eba67da admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:54.418739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-3eaa15f0-1db2-4753-88ef-7d828eba67da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:54.545080 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa15f0-1db2-4753-88ef-7d828eba67da admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:54.553444 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9269770-cc69-4754-87dc-2bf0f637cbc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:54.555123 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9269770-cc69-4754-87dc-2bf0f637cbc5 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:54.555952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9269770-cc69-4754-87dc-2bf0f637cbc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:54.556836 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9269770-cc69-4754-87dc-2bf0f637cbc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:54.583131 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9269770-cc69-4754-87dc-2bf0f637cbc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:54.586553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9269770-cc69-4754-87dc-2bf0f637cbc5 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:54.616140 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 929/1861] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:54 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:33:55.426428 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00d338-e8d2-453f-a2ee-95cf122ec03c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:55.591030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00d338-e8d2-453f-a2ee-95cf122ec03c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:55.599590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5393a83-ee7f-4c62-ab3f-1b24db0e6d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:55.600872 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5393a83-ee7f-4c62-ab3f-1b24db0e6d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:55.601635 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5393a83-ee7f-4c62-ab3f-1b24db0e6d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:55.602272 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5393a83-ee7f-4c62-ab3f-1b24db0e6d5b admin admin] PATH_INFO: `/v3/users/7469a8ed97604d45830fdbc5c3e5ff26` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:55.606179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5393a83-ee7f-4c62-ab3f-1b24db0e6d5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7469a8ed97604d45830fdbc5c3e5ff26)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:55.610072 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5393a83-ee7f-4c62-ab3f-1b24db0e6d5b admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:55.845841 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:55.990580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c5393a83-ee7f-4c62-ab3f-1b24db0e6d5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7469a8ed97604d45830fdbc5c3e5ff26'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:56.136656 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c5393a83-ee7f-4c62-ab3f-1b24db0e6d5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7469a8ed97604d45830fdbc5c3e5ff26'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:56.138571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:56.139844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c5393a83-ee7f-4c62-ab3f-1b24db0e6d5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7469a8ed97604d45830fdbc5c3e5ff26'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:56.140828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c5393a83-ee7f-4c62-ab3f-1b24db0e6d5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7469a8ed97604d45830fdbc5c3e5ff26'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:56.150132 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c5393a83-ee7f-4c62-ab3f-1b24db0e6d5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7469a8ed97604d45830fdbc5c3e5ff26'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:56.184166 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 933/1862] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:33:55 2022] DELETE /identity/v3/users/7469a8ed97604d45830fdbc5c3e5ff26 => generated 0 bytes in 759 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:56.190642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d229a-dd0d-44a8-b279-476f65aa2050 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:56.191184 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d229a-dd0d-44a8-b279-476f65aa2050 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:56.191694 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d229a-dd0d-44a8-b279-476f65aa2050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:56.225891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-586d229a-dd0d-44a8-b279-476f65aa2050 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:56.228810 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-586d229a-dd0d-44a8-b279-476f65aa2050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:56.368486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 930/1863] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:33:56.434074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5393a83-ee7f-4c62-ab3f-1b24db0e6d5b admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:56.436698 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c5393a83-ee7f-4c62-ab3f-1b24db0e6d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:56.590726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5393a83-ee7f-4c62-ab3f-1b24db0e6d5b admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:56.599898 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29788d89-a98d-4edb-b57a-f05465b9dcdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:56.601029 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29788d89-a98d-4edb-b57a-f05465b9dcdb service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:56.601757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29788d89-a98d-4edb-b57a-f05465b9dcdb service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:56.602265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29788d89-a98d-4edb-b57a-f05465b9dcdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:56.638218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29788d89-a98d-4edb-b57a-f05465b9dcdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:56.641466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29788d89-a98d-4edb-b57a-f05465b9dcdb service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:56.659343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 934/1864] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:33:57.544746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d229a-dd0d-44a8-b279-476f65aa2050 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:57.552052 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-586d229a-dd0d-44a8-b279-476f65aa2050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:57.588962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d229a-dd0d-44a8-b279-476f65aa2050 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:57.609058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac78818-576a-4814-8174-48c64fad485b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:57.610234 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac78818-576a-4814-8174-48c64fad485b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:57.611049 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac78818-576a-4814-8174-48c64fad485b admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:57.611806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac78818-576a-4814-8174-48c64fad485b admin admin] PATH_INFO: `/v3/users/c9c6ef63d1f041f6a7d320e47c864c0f` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:57.614451 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac78818-576a-4814-8174-48c64fad485b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9c6ef63d1f041f6a7d320e47c864c0f)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:57.619859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac78818-576a-4814-8174-48c64fad485b admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:57.741682 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29788d89-a98d-4edb-b57a-f05465b9dcdb service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:57.745157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-29788d89-a98d-4edb-b57a-f05465b9dcdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:57.806225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29788d89-a98d-4edb-b57a-f05465b9dcdb service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:57.818922 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:57.820618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:57.823813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:57.824926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb admin admin] PATH_INFO: `/v3/projects/2eb504b6396f49eeb69d34a491f9f93a` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:57.829158 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eb504b6396f49eeb69d34a491f9f93a)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:57.836522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:58.065743 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-1ac78818-576a-4814-8174-48c64fad485b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9c6ef63d1f041f6a7d320e47c864c0f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:58.095533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-1ac78818-576a-4814-8174-48c64fad485b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9c6ef63d1f041f6a7d320e47c864c0f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:58.096857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-1ac78818-576a-4814-8174-48c64fad485b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9c6ef63d1f041f6a7d320e47c864c0f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:58.113080 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-1ac78818-576a-4814-8174-48c64fad485b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9c6ef63d1f041f6a7d320e47c864c0f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:58.114301 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-1ac78818-576a-4814-8174-48c64fad485b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9c6ef63d1f041f6a7d320e47c864c0f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:58.129704 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eb504b6396f49eeb69d34a491f9f93a', 'request_id': 'req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:58.147580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eb504b6396f49eeb69d34a491f9f93a', 'request_id': 'req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:58.232722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 931/1865] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:33:57 2022] DELETE /identity/v3/users/c9c6ef63d1f041f6a7d320e47c864c0f => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:58.240124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aaa27-8854-42c4-9d21-4d3e0e242c37 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:58.240614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aaa27-8854-42c4-9d21-4d3e0e242c37 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:58.241363 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aaa27-8854-42c4-9d21-4d3e0e242c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:58.281179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-fe2aaa27-8854-42c4-9d21-4d3e0e242c37 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:33:58.283224 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-fe2aaa27-8854-42c4-9d21-4d3e0e242c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:58.376175 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eb504b6396f49eeb69d34a491f9f93a', 'request_id': 'req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:58.443604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:58.447553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb admin admin] The token cache is being invalidate because project 2eb504b6396f49eeb69d34a491f9f93a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:33:58.448001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:58.451178 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:58.458463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 935/1866] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:33:57 2022] DELETE /identity/v3/projects/2eb504b6396f49eeb69d34a491f9f93a => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:33:58.465753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 932/1867] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:33:58.467499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:58.473471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:58.678362 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:33:58.680109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aaa27-8854-42c4-9d21-4d3e0e242c37 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:58.680109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-fe2aaa27-8854-42c4-9d21-4d3e0e242c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:58.687597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222edb0a-cbb2-4bb2-bcbc-ded1f9b66cbb admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:58.699769 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec05e6-7c03-4749-838e-e888888364f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:58.700610 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec05e6-7c03-4749-838e-e888888364f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:58.700953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec05e6-7c03-4749-838e-e888888364f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:58.701288 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec05e6-7c03-4749-838e-e888888364f0 admin admin] PATH_INFO: `/v3/users/b0616d436e204326b706054adecd180c` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:58.704392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec05e6-7c03-4749-838e-e888888364f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0616d436e204326b706054adecd180c)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:58.709288 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec05e6-7c03-4749-838e-e888888364f0 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:58.787151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aaa27-8854-42c4-9d21-4d3e0e242c37 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:33:58.787151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37bed46-1936-44a2-bcf5-97f772843344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:33:58.787151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37bed46-1936-44a2-bcf5-97f772843344 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:33:58.787151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37bed46-1936-44a2-bcf5-97f772843344 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:33:58.787775 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37bed46-1936-44a2-bcf5-97f772843344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:33:58.787775 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f37bed46-1936-44a2-bcf5-97f772843344 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:58.789598 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37bed46-1936-44a2-bcf5-97f772843344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:33:58.791282 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37bed46-1936-44a2-bcf5-97f772843344 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:33:58.827326 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 933/1868] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:33:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:33:59.268252 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e8ec05e6-7c03-4749-838e-e888888364f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0616d436e204326b706054adecd180c'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:59.457208 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e8ec05e6-7c03-4749-838e-e888888364f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0616d436e204326b706054adecd180c'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:59.467466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e8ec05e6-7c03-4749-838e-e888888364f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0616d436e204326b706054adecd180c'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:59.475408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e8ec05e6-7c03-4749-838e-e888888364f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0616d436e204326b706054adecd180c'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:59.531868 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e8ec05e6-7c03-4749-838e-e888888364f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0616d436e204326b706054adecd180c'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:33:59.602331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 936/1869] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:33:58 2022] DELETE /identity/v3/users/b0616d436e204326b706054adecd180c => generated 0 bytes in 1137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:33:59.818412 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37bed46-1936-44a2-bcf5-97f772843344 service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:33:59.819204 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f37bed46-1936-44a2-bcf5-97f772843344 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:33:59.844660 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37bed46-1936-44a2-bcf5-97f772843344 service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:00.014840 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:00.020072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:00.020505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:00.021053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7 admin admin] PATH_INFO: `/v3/projects/bf386efa76c44687a401e3559f157fc5` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:00.027896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf386efa76c44687a401e3559f157fc5)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:00.038570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:00.367894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf386efa76c44687a401e3559f157fc5', 'request_id': 'req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:00.401649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf386efa76c44687a401e3559f157fc5', 'request_id': 'req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:00.522031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf386efa76c44687a401e3559f157fc5', 'request_id': 'req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:00.570814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7 admin admin] The token cache is being invalidate because project bf386efa76c44687a401e3559f157fc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:00.573774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:00.578473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:00.590168 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 934/1870] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:33:59 2022] DELETE /identity/v3/projects/bf386efa76c44687a401e3559f157fc5 => generated 0 bytes in 776 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:00.596641 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec05e6-7c03-4749-838e-e888888364f0 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:00.601154 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e8ec05e6-7c03-4749-838e-e888888364f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:00.689037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec05e6-7c03-4749-838e-e888888364f0 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:00.860919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:00.860919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:00.934663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b518ab-fa24-463d-9fc2-db59af7144d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:00.940369 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b518ab-fa24-463d-9fc2-db59af7144d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:00.941644 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b518ab-fa24-463d-9fc2-db59af7144d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:00.942439 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b518ab-fa24-463d-9fc2-db59af7144d5 admin admin] PATH_INFO: `/v3/users/3f998265a2e1458f9d40fd519eb38e9e` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:00.953177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b518ab-fa24-463d-9fc2-db59af7144d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f998265a2e1458f9d40fd519eb38e9e)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:00.959229 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b518ab-fa24-463d-9fc2-db59af7144d5 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:01.109948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd3c2a-4245-4d6c-9fdf-f2e15f1feff7 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:01.130264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8130d6-fbe2-4e76-a90f-0095906dc4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:01.131888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8130d6-fbe2-4e76-a90f-0095906dc4dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:01.135827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8130d6-fbe2-4e76-a90f-0095906dc4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:01.135827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8130d6-fbe2-4e76-a90f-0095906dc4dd admin admin] PATH_INFO: `/v3/projects/5419f47d6d024e4d8771f6e99bb5f14c` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:01.137405 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8130d6-fbe2-4e76-a90f-0095906dc4dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5419f47d6d024e4d8771f6e99bb5f14c)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:01.141571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8130d6-fbe2-4e76-a90f-0095906dc4dd admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:01.326044 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-84b518ab-fa24-463d-9fc2-db59af7144d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f998265a2e1458f9d40fd519eb38e9e'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:01.334470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ca8130d6-fbe2-4e76-a90f-0095906dc4dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5419f47d6d024e4d8771f6e99bb5f14c', 'request_id': 'req-ca8130d6-fbe2-4e76-a90f-0095906dc4dd'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:01.348160 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ca8130d6-fbe2-4e76-a90f-0095906dc4dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5419f47d6d024e4d8771f6e99bb5f14c', 'request_id': 'req-ca8130d6-fbe2-4e76-a90f-0095906dc4dd'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:01.479304 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-84b518ab-fa24-463d-9fc2-db59af7144d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f998265a2e1458f9d40fd519eb38e9e'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:01.480528 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-84b518ab-fa24-463d-9fc2-db59af7144d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f998265a2e1458f9d40fd519eb38e9e'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:01.481711 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-84b518ab-fa24-463d-9fc2-db59af7144d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f998265a2e1458f9d40fd519eb38e9e'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:01.502054 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-84b518ab-fa24-463d-9fc2-db59af7144d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f998265a2e1458f9d40fd519eb38e9e'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:01.564003 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ca8130d6-fbe2-4e76-a90f-0095906dc4dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5419f47d6d024e4d8771f6e99bb5f14c', 'request_id': 'req-ca8130d6-fbe2-4e76-a90f-0095906dc4dd'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:01.571541 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 937/1871] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:00 2022] DELETE /identity/v3/users/3f998265a2e1458f9d40fd519eb38e9e => generated 0 bytes in 976 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:01.575458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b518ab-fa24-463d-9fc2-db59af7144d5 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:01.580194 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-84b518ab-fa24-463d-9fc2-db59af7144d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:01.606593 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ca8130d6-fbe2-4e76-a90f-0095906dc4dd admin admin] The token cache is being invalidate because project 5419f47d6d024e4d8771f6e99bb5f14c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:01.607046 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ca8130d6-fbe2-4e76-a90f-0095906dc4dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:01.609378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ca8130d6-fbe2-4e76-a90f-0095906dc4dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:01.622016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 935/1872] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:00 2022] DELETE /identity/v3/projects/5419f47d6d024e4d8771f6e99bb5f14c => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:01.629711 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25781fc4-7fda-4fbf-86b9-35e638e5992a None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:01.630156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25781fc4-7fda-4fbf-86b9-35e638e5992a None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:01.630675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25781fc4-7fda-4fbf-86b9-35e638e5992a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:01.695606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-25781fc4-7fda-4fbf-86b9-35e638e5992a None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:01.697956 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-25781fc4-7fda-4fbf-86b9-35e638e5992a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:01.765003 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:01.836579 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 936/1873] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:01.856004 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c682ced-cb97-44ad-92cc-b1f88ccfc7cd None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:01.856894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c682ced-cb97-44ad-92cc-b1f88ccfc7cd None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:01.856894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c682ced-cb97-44ad-92cc-b1f88ccfc7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:01.875430 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b518ab-fa24-463d-9fc2-db59af7144d5 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:01.883414 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c82fec-d338-4d65-9a78-43d6f0c69f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:01.885395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c82fec-d338-4d65-9a78-43d6f0c69f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:01.886083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c82fec-d338-4d65-9a78-43d6f0c69f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:01.886385 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c82fec-d338-4d65-9a78-43d6f0c69f7f admin admin] PATH_INFO: `/v3/users/3a9a1057fff246d7b41234c073cdb914` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:01.900872 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c82fec-d338-4d65-9a78-43d6f0c69f7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a9a1057fff246d7b41234c073cdb914)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:01.904870 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c82fec-d338-4d65-9a78-43d6f0c69f7f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:01.930116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-4c682ced-cb97-44ad-92cc-b1f88ccfc7cd None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:01.943409 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-4c682ced-cb97-44ad-92cc-b1f88ccfc7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:01.989040 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 937/1874] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:34:02.001276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:02.006485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c682ced-cb97-44ad-92cc-b1f88ccfc7cd None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:02.022538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c682ced-cb97-44ad-92cc-b1f88ccfc7cd None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:02.062674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d390d06-d496-4a1f-8c60-2f04dec1d675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:02.063836 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d390d06-d496-4a1f-8c60-2f04dec1d675 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:02.068824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d390d06-d496-4a1f-8c60-2f04dec1d675 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:02.068824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d390d06-d496-4a1f-8c60-2f04dec1d675 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:02.068824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d390d06-d496-4a1f-8c60-2f04dec1d675 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:34:02.068824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d390d06-d496-4a1f-8c60-2f04dec1d675 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:02.068824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d390d06-d496-4a1f-8c60-2f04dec1d675 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:02.111491 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 938/1875] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:34:02.117804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d390d06-d496-4a1f-8c60-2f04dec1d675 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:02.127934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c1c82fec-d338-4d65-9a78-43d6f0c69f7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a9a1057fff246d7b41234c073cdb914'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:02.157684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d390d06-d496-4a1f-8c60-2f04dec1d675 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:02.192095 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631e8e7-73d0-4481-be70-369ea0605db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:02.192095 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631e8e7-73d0-4481-be70-369ea0605db7 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:02.192095 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631e8e7-73d0-4481-be70-369ea0605db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:02.192095 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631e8e7-73d0-4481-be70-369ea0605db7 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:02.192795 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631e8e7-73d0-4481-be70-369ea0605db7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:02.192795 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631e8e7-73d0-4481-be70-369ea0605db7 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:02.250375 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c1c82fec-d338-4d65-9a78-43d6f0c69f7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a9a1057fff246d7b41234c073cdb914'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:02.252958 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c1c82fec-d338-4d65-9a78-43d6f0c69f7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a9a1057fff246d7b41234c073cdb914'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:02.252958 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c1c82fec-d338-4d65-9a78-43d6f0c69f7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a9a1057fff246d7b41234c073cdb914'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:02.268559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c1c82fec-d338-4d65-9a78-43d6f0c69f7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a9a1057fff246d7b41234c073cdb914'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:02.310315 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 938/1876] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:01 2022] DELETE /identity/v3/users/3a9a1057fff246d7b41234c073cdb914 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:02.356993 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 939/1877] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:02 2022] POST /identity/v3/projects => generated 385 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:34:02.362501 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c82fec-d338-4d65-9a78-43d6f0c69f7f admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:02.364516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c1c82fec-d338-4d65-9a78-43d6f0c69f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:02.511614 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c82fec-d338-4d65-9a78-43d6f0c69f7f admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:02.645754 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cfef9-da83-4b30-be75-5e05d8b9a0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:02.646443 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cfef9-da83-4b30-be75-5e05d8b9a0e5 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:02.646923 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cfef9-da83-4b30-be75-5e05d8b9a0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:02.647132 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cfef9-da83-4b30-be75-5e05d8b9a0e5 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:02.647896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cfef9-da83-4b30-be75-5e05d8b9a0e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:02.649947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cfef9-da83-4b30-be75-5e05d8b9a0e5 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:02.762301 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631e8e7-73d0-4481-be70-369ea0605db7 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:02.765969 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9631e8e7-73d0-4481-be70-369ea0605db7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:02.798452 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631e8e7-73d0-4481-be70-369ea0605db7 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:02.816517 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069fd2f-3207-4019-851e-2fcbf7772e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:02.819108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069fd2f-3207-4019-851e-2fcbf7772e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:02.819687 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069fd2f-3207-4019-851e-2fcbf7772e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:02.826759 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069fd2f-3207-4019-851e-2fcbf7772e43 admin admin] PATH_INFO: `/v3/projects/c962b174c7004e5eac678e8e96e81560` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:02.829543 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 939/1878] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:02 2022] POST /identity/v3/users => generated 293 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:02.834865 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cfef9-da83-4b30-be75-5e05d8b9a0e5 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:02.848723 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cfef9-da83-4b30-be75-5e05d8b9a0e5 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:02.850524 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069fd2f-3207-4019-851e-2fcbf7772e43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c962b174c7004e5eac678e8e96e81560)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:02.852991 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069fd2f-3207-4019-851e-2fcbf7772e43 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:02.855670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe08503-0a40-457e-8619-660ebadf8b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:02.856427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe08503-0a40-457e-8619-660ebadf8b9e admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:02.857022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe08503-0a40-457e-8619-660ebadf8b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:02.857208 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe08503-0a40-457e-8619-660ebadf8b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:02.858371 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe08503-0a40-457e-8619-660ebadf8b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:02.860739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe08503-0a40-457e-8619-660ebadf8b9e admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:02.877122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 940/1879] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:02.883764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe08503-0a40-457e-8619-660ebadf8b9e admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:02.903218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe08503-0a40-457e-8619-660ebadf8b9e admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:02.911522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a915a4c-38eb-4f77-9f91-c19ce935a271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:02.911522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a915a4c-38eb-4f77-9f91-c19ce935a271 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:02.911522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a915a4c-38eb-4f77-9f91-c19ce935a271 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:02.912165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a915a4c-38eb-4f77-9f91-c19ce935a271 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:02.912868 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a915a4c-38eb-4f77-9f91-c19ce935a271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:02.914962 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a915a4c-38eb-4f77-9f91-c19ce935a271 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:02.944828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 941/1880] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:02.951412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a915a4c-38eb-4f77-9f91-c19ce935a271 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:02.966434 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a915a4c-38eb-4f77-9f91-c19ce935a271 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:02.984069 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd23f92-dd87-46f1-9b95-0ad6822920d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:02.984069 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd23f92-dd87-46f1-9b95-0ad6822920d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:02.984069 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd23f92-dd87-46f1-9b95-0ad6822920d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:02.984682 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd23f92-dd87-46f1-9b95-0ad6822920d2 admin admin] PATH_INFO: `/v3/projects/2aab174a71254e998937b1fe64bea701/users/c8c7090a5f074137b3de0794f9275b43/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:03.017245 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd23f92-dd87-46f1-9b95-0ad6822920d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aab174a71254e998937b1fe64bea701, user_id=c8c7090a5f074137b3de0794f9275b43, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:03.021440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd23f92-dd87-46f1-9b95-0ad6822920d2 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:03.145456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 942/1881] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:02 2022] PUT /identity/v3/projects/2aab174a71254e998937b1fe64bea701/users/c8c7090a5f074137b3de0794f9275b43/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:03.145456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd23f92-dd87-46f1-9b95-0ad6822920d2 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:03.157971 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0fd23f92-dd87-46f1-9b95-0ad6822920d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:03.207386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3069fd2f-3207-4019-851e-2fcbf7772e43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c962b174c7004e5eac678e8e96e81560', 'request_id': 'req-3069fd2f-3207-4019-851e-2fcbf7772e43'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:03.226995 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3069fd2f-3207-4019-851e-2fcbf7772e43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c962b174c7004e5eac678e8e96e81560', 'request_id': 'req-3069fd2f-3207-4019-851e-2fcbf7772e43'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:03.321266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3069fd2f-3207-4019-851e-2fcbf7772e43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c962b174c7004e5eac678e8e96e81560', 'request_id': 'req-3069fd2f-3207-4019-851e-2fcbf7772e43'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:03.321520 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd23f92-dd87-46f1-9b95-0ad6822920d2 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:03.343373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3069fd2f-3207-4019-851e-2fcbf7772e43 admin admin] The token cache is being invalidate because project c962b174c7004e5eac678e8e96e81560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:03.343910 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3069fd2f-3207-4019-851e-2fcbf7772e43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:03.345342 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3069fd2f-3207-4019-851e-2fcbf7772e43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:03.346454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 940/1882] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:02 2022] DELETE /identity/v3/projects/c962b174c7004e5eac678e8e96e81560 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:03.354831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c49a6-52f0-4062-8417-94552f1359aa None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:03.354831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c49a6-52f0-4062-8417-94552f1359aa None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:03.354831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c49a6-52f0-4062-8417-94552f1359aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:03.404116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c44c49a6-52f0-4062-8417-94552f1359aa None None] MFA Rules not processed for user `c8c7090a5f074137b3de0794f9275b43`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:03.405415 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c44c49a6-52f0-4062-8417-94552f1359aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:03.411504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f492cb8-4121-43ef-b981-5464e0c2128f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:03.412232 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f492cb8-4121-43ef-b981-5464e0c2128f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:03.412516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f492cb8-4121-43ef-b981-5464e0c2128f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:03.412795 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f492cb8-4121-43ef-b981-5464e0c2128f admin admin] PATH_INFO: `/v3/projects/0961542fe2624e599ee3fd7d4fb52f5b` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:03.414270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f492cb8-4121-43ef-b981-5464e0c2128f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0961542fe2624e599ee3fd7d4fb52f5b)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:03.416756 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f492cb8-4121-43ef-b981-5464e0c2128f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:03.464299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:03.507452 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 941/1883] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:03.508911 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c49a6-52f0-4062-8417-94552f1359aa None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:03.511401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c44c49a6-52f0-4062-8417-94552f1359aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:03.528661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c49a6-52f0-4062-8417-94552f1359aa None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:03.538137 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61a16d-1b17-4ccc-bcfe-756c7904dbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:03.539326 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61a16d-1b17-4ccc-bcfe-756c7904dbc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:03.540489 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61a16d-1b17-4ccc-bcfe-756c7904dbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:03.541841 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61a16d-1b17-4ccc-bcfe-756c7904dbc0 admin admin] PATH_INFO: `/v3/users/7e5f1078006549c4b93a175f733b5b3a` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:03.544797 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61a16d-1b17-4ccc-bcfe-756c7904dbc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e5f1078006549c4b93a175f733b5b3a)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:03.548072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61a16d-1b17-4ccc-bcfe-756c7904dbc0 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:03.563442 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9f492cb8-4121-43ef-b981-5464e0c2128f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0961542fe2624e599ee3fd7d4fb52f5b', 'request_id': 'req-9f492cb8-4121-43ef-b981-5464e0c2128f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:03.571495 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9f492cb8-4121-43ef-b981-5464e0c2128f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0961542fe2624e599ee3fd7d4fb52f5b', 'request_id': 'req-9f492cb8-4121-43ef-b981-5464e0c2128f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:03.663750 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9f492cb8-4121-43ef-b981-5464e0c2128f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0961542fe2624e599ee3fd7d4fb52f5b', 'request_id': 'req-9f492cb8-4121-43ef-b981-5464e0c2128f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:03.684396 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9f492cb8-4121-43ef-b981-5464e0c2128f admin admin] The token cache is being invalidate because project 0961542fe2624e599ee3fd7d4fb52f5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:03.684980 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9f492cb8-4121-43ef-b981-5464e0c2128f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:03.686072 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9f492cb8-4121-43ef-b981-5464e0c2128f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:03.688733 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 943/1884] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:03 2022] DELETE /identity/v3/projects/0961542fe2624e599ee3fd7d4fb52f5b => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:03.689865 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f492cb8-4121-43ef-b981-5464e0c2128f admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:03.691716 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9f492cb8-4121-43ef-b981-5464e0c2128f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:03.801516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f492cb8-4121-43ef-b981-5464e0c2128f admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:03.810200 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28142c-cb15-4426-8219-1deac39062a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:03.811722 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28142c-cb15-4426-8219-1deac39062a8 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:03.812219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28142c-cb15-4426-8219-1deac39062a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:03.812754 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28142c-cb15-4426-8219-1deac39062a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:03.815530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0a28142c-cb15-4426-8219-1deac39062a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:03.874387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-6a61a16d-1b17-4ccc-bcfe-756c7904dbc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e5f1078006549c4b93a175f733b5b3a'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:03.894641 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-6a61a16d-1b17-4ccc-bcfe-756c7904dbc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e5f1078006549c4b93a175f733b5b3a'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:03.895928 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-6a61a16d-1b17-4ccc-bcfe-756c7904dbc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e5f1078006549c4b93a175f733b5b3a'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:03.908384 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-6a61a16d-1b17-4ccc-bcfe-756c7904dbc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e5f1078006549c4b93a175f733b5b3a'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:03.909150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-6a61a16d-1b17-4ccc-bcfe-756c7904dbc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e5f1078006549c4b93a175f733b5b3a'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:03.925066 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28142c-cb15-4426-8219-1deac39062a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:03.925066 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28142c-cb15-4426-8219-1deac39062a8 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:03.937261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 944/1885] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:03.938878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7752fc9-263b-4583-9d03-3f0b07dc57bb None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:03.939286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7752fc9-263b-4583-9d03-3f0b07dc57bb None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:03.939519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7752fc9-263b-4583-9d03-3f0b07dc57bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:03.964137 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-c7752fc9-263b-4583-9d03-3f0b07dc57bb None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:03.965466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c7752fc9-263b-4583-9d03-3f0b07dc57bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:04.021025 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 942/1886] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:03 2022] DELETE /identity/v3/users/7e5f1078006549c4b93a175f733b5b3a => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:04.060719 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 945/1887] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:04.063253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b9604-e5d1-4e9a-8de0-859d20ee9071 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:04.063841 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b9604-e5d1-4e9a-8de0-859d20ee9071 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:04.064411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b9604-e5d1-4e9a-8de0-859d20ee9071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:04.115016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c64b9604-e5d1-4e9a-8de0-859d20ee9071 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:04.120802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c64b9604-e5d1-4e9a-8de0-859d20ee9071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:04.277072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 943/1888] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:04.283346 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7752fc9-263b-4583-9d03-3f0b07dc57bb None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:04.302738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7752fc9-263b-4583-9d03-3f0b07dc57bb None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:04.316748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c89a4f-bfe3-43b1-b610-8f5a5b2e8069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:04.319956 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c89a4f-bfe3-43b1-b610-8f5a5b2e8069 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:04.324547 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c89a4f-bfe3-43b1-b610-8f5a5b2e8069 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:04.324547 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c89a4f-bfe3-43b1-b610-8f5a5b2e8069 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:04.324547 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c89a4f-bfe3-43b1-b610-8f5a5b2e8069 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:34:04.324547 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c89a4f-bfe3-43b1-b610-8f5a5b2e8069 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:04.326462 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c89a4f-bfe3-43b1-b610-8f5a5b2e8069 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:04.347767 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 946/1889] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:34:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:34:04.355131 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:04.355435 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b9604-e5d1-4e9a-8de0-859d20ee9071 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:04.375585 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b9604-e5d1-4e9a-8de0-859d20ee9071 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:04.382772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67674193-0cd0-421d-8cb8-1b6c4df86c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:04.383793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67674193-0cd0-421d-8cb8-1b6c4df86c4d admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:04.384217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67674193-0cd0-421d-8cb8-1b6c4df86c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:04.384598 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67674193-0cd0-421d-8cb8-1b6c4df86c4d admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:04.386200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67674193-0cd0-421d-8cb8-1b6c4df86c4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:04.388422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67674193-0cd0-421d-8cb8-1b6c4df86c4d admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:04.518360 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 944/1890] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:04 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:34:04.523687 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c89a4f-bfe3-43b1-b610-8f5a5b2e8069 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:04.554060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c89a4f-bfe3-43b1-b610-8f5a5b2e8069 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:04.633678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67674193-0cd0-421d-8cb8-1b6c4df86c4d admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:04.639917 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-67674193-0cd0-421d-8cb8-1b6c4df86c4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:04.702503 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5555a-5447-44a6-b6de-f0b926712a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:04.704310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5555a-5447-44a6-b6de-f0b926712a3f admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:04.704763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5555a-5447-44a6-b6de-f0b926712a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:04.705214 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5555a-5447-44a6-b6de-f0b926712a3f admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:04.706513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5555a-5447-44a6-b6de-f0b926712a3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:04.709895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5555a-5447-44a6-b6de-f0b926712a3f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:04.779114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67674193-0cd0-421d-8cb8-1b6c4df86c4d admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:04.784803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5ca9b-8897-4dc3-819a-1c02e38f68f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:04.785466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5ca9b-8897-4dc3-819a-1c02e38f68f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:04.785745 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5ca9b-8897-4dc3-819a-1c02e38f68f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:04.786023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5ca9b-8897-4dc3-819a-1c02e38f68f3 admin admin] PATH_INFO: `/v3/projects/55dfc66bdba247998846b11a4cbabe64` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:04.787282 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5ca9b-8897-4dc3-819a-1c02e38f68f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55dfc66bdba247998846b11a4cbabe64)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:04.789276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5ca9b-8897-4dc3-819a-1c02e38f68f3 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:04.862145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 947/1891] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:04 2022] POST /identity/v3/users => generated 290 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:04.866307 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5555a-5447-44a6-b6de-f0b926712a3f admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:04.883265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5555a-5447-44a6-b6de-f0b926712a3f admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:04.891959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3f95d-1922-474a-ab26-3ee5cb892b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:04.893738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3f95d-1922-474a-ab26-3ee5cb892b24 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:04.893738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3f95d-1922-474a-ab26-3ee5cb892b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:04.894188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3f95d-1922-474a-ab26-3ee5cb892b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:04.895664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3f95d-1922-474a-ab26-3ee5cb892b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:04.898049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3f95d-1922-474a-ab26-3ee5cb892b24 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:04.915487 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 948/1892] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:34:04.920161 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3f95d-1922-474a-ab26-3ee5cb892b24 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:04.931092 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3f95d-1922-474a-ab26-3ee5cb892b24 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:04.940707 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b84858-9f13-4893-910c-f2613cadc7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:04.941857 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b84858-9f13-4893-910c-f2613cadc7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:04.942225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b84858-9f13-4893-910c-f2613cadc7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:04.944984 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b84858-9f13-4893-910c-f2613cadc7b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:04.944984 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b84858-9f13-4893-910c-f2613cadc7b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:04.948987 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b84858-9f13-4893-910c-f2613cadc7b6 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:04.966745 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 949/1893] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:04.967802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-06a5ca9b-8897-4dc3-819a-1c02e38f68f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55dfc66bdba247998846b11a4cbabe64', 'request_id': 'req-06a5ca9b-8897-4dc3-819a-1c02e38f68f3'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:04.968365 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b84858-9f13-4893-910c-f2613cadc7b6 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:04.970661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-06a5ca9b-8897-4dc3-819a-1c02e38f68f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55dfc66bdba247998846b11a4cbabe64', 'request_id': 'req-06a5ca9b-8897-4dc3-819a-1c02e38f68f3'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:04.980871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b84858-9f13-4893-910c-f2613cadc7b6 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:05.057514 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb315c-4ae6-4882-bb3b-fc343d45567a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:05.079489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb315c-4ae6-4882-bb3b-fc343d45567a admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:05.079489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb315c-4ae6-4882-bb3b-fc343d45567a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:05.079489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb315c-4ae6-4882-bb3b-fc343d45567a admin admin] PATH_INFO: `/v3/projects/e1b80031cca5465695bd0724b37c9fe3/users/d6140074c2014625b1b1f5402bea9457/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:05.084775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb315c-4ae6-4882-bb3b-fc343d45567a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1b80031cca5465695bd0724b37c9fe3, user_id=d6140074c2014625b1b1f5402bea9457, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:05.088516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb315c-4ae6-4882-bb3b-fc343d45567a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:05.097717 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-06a5ca9b-8897-4dc3-819a-1c02e38f68f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55dfc66bdba247998846b11a4cbabe64', 'request_id': 'req-06a5ca9b-8897-4dc3-819a-1c02e38f68f3'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:05.115886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-06a5ca9b-8897-4dc3-819a-1c02e38f68f3 admin admin] The token cache is being invalidate because project 55dfc66bdba247998846b11a4cbabe64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:05.116186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-06a5ca9b-8897-4dc3-819a-1c02e38f68f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:05.116946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-06a5ca9b-8897-4dc3-819a-1c02e38f68f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:05.118738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 945/1894] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:04 2022] DELETE /identity/v3/projects/55dfc66bdba247998846b11a4cbabe64 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:05.126153 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7e108-4064-49e9-8294-a9488e524f1f None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:05.126748 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7e108-4064-49e9-8294-a9488e524f1f None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:05.127214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7e108-4064-49e9-8294-a9488e524f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:05.163642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-37c7e108-4064-49e9-8294-a9488e524f1f None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:05.165929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-37c7e108-4064-49e9-8294-a9488e524f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:05.177688 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 946/1895] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:05.194476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1983b-a285-4178-b7f3-26ed4c5ddb38 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:05.194476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1983b-a285-4178-b7f3-26ed4c5ddb38 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:05.194476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1983b-a285-4178-b7f3-26ed4c5ddb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:05.198016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 950/1896] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:04 2022] PUT /identity/v3/projects/e1b80031cca5465695bd0724b37c9fe3/users/d6140074c2014625b1b1f5402bea9457/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:05.211038 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8344a0-0315-4f2e-a6d9-9d74a9d1f142 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:05.211625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8344a0-0315-4f2e-a6d9-9d74a9d1f142 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:05.212291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8344a0-0315-4f2e-a6d9-9d74a9d1f142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:05.245995 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-3cd1983b-a285-4178-b7f3-26ed4c5ddb38 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:05.247886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-3cd1983b-a285-4178-b7f3-26ed4c5ddb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:05.257345 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:05.267825 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-cd8344a0-0315-4f2e-a6d9-9d74a9d1f142 None None] MFA Rules not processed for user `d6140074c2014625b1b1f5402bea9457`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:05.270510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-cd8344a0-0315-4f2e-a6d9-9d74a9d1f142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:05.387306 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 951/1897] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:05.392853 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8344a0-0315-4f2e-a6d9-9d74a9d1f142 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:05.395041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-cd8344a0-0315-4f2e-a6d9-9d74a9d1f142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:05.415470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 947/1898] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:34:05.424431 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd1983b-a285-4178-b7f3-26ed4c5ddb38 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:05.441700 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd1983b-a285-4178-b7f3-26ed4c5ddb38 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:05.457709 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c11e50-96f4-4d5f-8587-6b359e9db297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:05.468040 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c11e50-96f4-4d5f-8587-6b359e9db297 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:05.468615 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c11e50-96f4-4d5f-8587-6b359e9db297 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:05.469148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c11e50-96f4-4d5f-8587-6b359e9db297 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:05.470665 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c11e50-96f4-4d5f-8587-6b359e9db297 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:34:05.471322 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c11e50-96f4-4d5f-8587-6b359e9db297 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:05.474566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c11e50-96f4-4d5f-8587-6b359e9db297 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:05.503028 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 948/1899] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:34:05.510561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c11e50-96f4-4d5f-8587-6b359e9db297 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:05.556400 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c11e50-96f4-4d5f-8587-6b359e9db297 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:05.574280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02deda73-3795-4936-9656-e41e812d282c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:05.576144 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deda73-3795-4936-9656-e41e812d282c admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:05.576639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deda73-3795-4936-9656-e41e812d282c admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:05.577139 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deda73-3795-4936-9656-e41e812d282c admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:05.578575 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02deda73-3795-4936-9656-e41e812d282c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:05.581535 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02deda73-3795-4936-9656-e41e812d282c admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:05.590910 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8344a0-0315-4f2e-a6d9-9d74a9d1f142 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:05.599404 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfa97a-b178-4f97-9005-ba684ba52c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:05.608943 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfa97a-b178-4f97-9005-ba684ba52c69 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:05.608943 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfa97a-b178-4f97-9005-ba684ba52c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:05.608943 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfa97a-b178-4f97-9005-ba684ba52c69 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:05.608943 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfa97a-b178-4f97-9005-ba684ba52c69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:05.608943 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfa97a-b178-4f97-9005-ba684ba52c69 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:05.748667 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 949/1900] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:05 2022] POST /identity/v3/projects => generated 385 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:05.754942 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02deda73-3795-4936-9656-e41e812d282c admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:05.818408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 952/1901] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:05 2022] POST /identity/v3/projects => generated 377 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:34:05.826568 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfa97a-b178-4f97-9005-ba684ba52c69 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:05.902956 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02deda73-3795-4936-9656-e41e812d282c admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:05.959302 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfa97a-b178-4f97-9005-ba684ba52c69 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:05.969794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d94b79-a9fc-4afa-a1f2-a5d062822520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:05.971799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d94b79-a9fc-4afa-a1f2-a5d062822520 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:05.972223 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d94b79-a9fc-4afa-a1f2-a5d062822520 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:05.972552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d94b79-a9fc-4afa-a1f2-a5d062822520 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:05.973377 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d94b79-a9fc-4afa-a1f2-a5d062822520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:05.976380 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d94b79-a9fc-4afa-a1f2-a5d062822520 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:06.028618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470badcf-e2c9-4ec5-9acf-329693505eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:06.029612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470badcf-e2c9-4ec5-9acf-329693505eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:06.030260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470badcf-e2c9-4ec5-9acf-329693505eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:06.030798 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470badcf-e2c9-4ec5-9acf-329693505eb9 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:06.031526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470badcf-e2c9-4ec5-9acf-329693505eb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:06.033624 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470badcf-e2c9-4ec5-9acf-329693505eb9 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:06.161105 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 953/1902] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:05 2022] POST /identity/v3/users => generated 295 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:06.163385 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d94b79-a9fc-4afa-a1f2-a5d062822520 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:06.187743 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d94b79-a9fc-4afa-a1f2-a5d062822520 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:06.213067 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 950/1903] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:05 2022] POST /identity/v3/users => generated 293 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:06.221024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470badcf-e2c9-4ec5-9acf-329693505eb9 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:06.231989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f786cc4-e7ee-46df-ae81-9871e4487dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:06.233813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f786cc4-e7ee-46df-ae81-9871e4487dca admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:06.234711 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f786cc4-e7ee-46df-ae81-9871e4487dca admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:06.235580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f786cc4-e7ee-46df-ae81-9871e4487dca admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:06.238276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f786cc4-e7ee-46df-ae81-9871e4487dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:06.242337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f786cc4-e7ee-46df-ae81-9871e4487dca admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:06.247517 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470badcf-e2c9-4ec5-9acf-329693505eb9 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:06.259450 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131d95d-240c-4962-a511-f1b0f8cbad54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:06.261191 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131d95d-240c-4962-a511-f1b0f8cbad54 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:06.262619 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131d95d-240c-4962-a511-f1b0f8cbad54 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:06.262619 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131d95d-240c-4962-a511-f1b0f8cbad54 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:06.263239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131d95d-240c-4962-a511-f1b0f8cbad54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:06.268153 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131d95d-240c-4962-a511-f1b0f8cbad54 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:06.284674 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 954/1904] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:06.289716 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f786cc4-e7ee-46df-ae81-9871e4487dca admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:06.303751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 951/1905] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:06.312098 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131d95d-240c-4962-a511-f1b0f8cbad54 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:06.323278 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f786cc4-e7ee-46df-ae81-9871e4487dca admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:06.329226 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131d95d-240c-4962-a511-f1b0f8cbad54 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:06.339723 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f62a1-2e51-470f-8a11-f6ad0585923f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:06.342240 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f62a1-2e51-470f-8a11-f6ad0585923f admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:06.342654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f24bf0-e475-4f3a-aed9-9fbabd3da259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:06.343215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f62a1-2e51-470f-8a11-f6ad0585923f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:06.343775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f62a1-2e51-470f-8a11-f6ad0585923f admin admin] PATH_INFO: `/v3/projects/328f9e9600ca45098513424016a15eb3/users/a554958a6dff4169927a0ced159596e4/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:06.348281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f24bf0-e475-4f3a-aed9-9fbabd3da259 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:06.348281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f24bf0-e475-4f3a-aed9-9fbabd3da259 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:06.348281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f24bf0-e475-4f3a-aed9-9fbabd3da259 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:06.348281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f24bf0-e475-4f3a-aed9-9fbabd3da259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:06.349898 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f24bf0-e475-4f3a-aed9-9fbabd3da259 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:06.382522 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 952/1906] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:06.384268 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f24bf0-e475-4f3a-aed9-9fbabd3da259 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:06.389216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f62a1-2e51-470f-8a11-f6ad0585923f admin admin] RBAC: Authorizing `identity:create_grant(project_id=328f9e9600ca45098513424016a15eb3, user_id=a554958a6dff4169927a0ced159596e4, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:06.395342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f62a1-2e51-470f-8a11-f6ad0585923f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:06.417573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f24bf0-e475-4f3a-aed9-9fbabd3da259 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:06.427711 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61440f9d-d763-4046-b97e-da5ae1087ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:06.429950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61440f9d-d763-4046-b97e-da5ae1087ce6 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:06.430923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61440f9d-d763-4046-b97e-da5ae1087ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:06.432388 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61440f9d-d763-4046-b97e-da5ae1087ce6 admin admin] PATH_INFO: `/v3/projects/b0d492f6eb8549978df1ff49b8b5a186/users/d2104f97b9cc41d2a61d34e64a5d19f9/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:06.464862 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61440f9d-d763-4046-b97e-da5ae1087ce6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0d492f6eb8549978df1ff49b8b5a186, user_id=d2104f97b9cc41d2a61d34e64a5d19f9, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:06.471357 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61440f9d-d763-4046-b97e-da5ae1087ce6 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:06.550685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 955/1907] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:06 2022] PUT /identity/v3/projects/328f9e9600ca45098513424016a15eb3/users/a554958a6dff4169927a0ced159596e4/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:06.557412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f62a1-2e51-470f-8a11-f6ad0585923f admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:06.639059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 953/1908] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:06 2022] PUT /identity/v3/projects/b0d492f6eb8549978df1ff49b8b5a186/users/d2104f97b9cc41d2a61d34e64a5d19f9/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:06.672671 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2e2a9-8e1d-4b06-b5a6-b37b002af0b1 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:06.673359 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2e2a9-8e1d-4b06-b5a6-b37b002af0b1 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:06.673930 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2e2a9-8e1d-4b06-b5a6-b37b002af0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:06.722793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-f2b2e2a9-8e1d-4b06-b5a6-b37b002af0b1 None None] MFA Rules not processed for user `d2104f97b9cc41d2a61d34e64a5d19f9`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:06.724361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f2b2e2a9-8e1d-4b06-b5a6-b37b002af0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:06.737520 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f62a1-2e51-470f-8a11-f6ad0585923f admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:06.857881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 954/1909] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:06.864911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d48034-cced-4e3d-8deb-4d15ca9034df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:06.867447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d48034-cced-4e3d-8deb-4d15ca9034df admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:06.868099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d48034-cced-4e3d-8deb-4d15ca9034df admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:06.868719 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d48034-cced-4e3d-8deb-4d15ca9034df admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:06.871115 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d48034-cced-4e3d-8deb-4d15ca9034df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:06.875832 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d48034-cced-4e3d-8deb-4d15ca9034df admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:06.876113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2e2a9-8e1d-4b06-b5a6-b37b002af0b1 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:06.878898 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f2b2e2a9-8e1d-4b06-b5a6-b37b002af0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:06.901487 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 956/1910] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:06.902554 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d48034-cced-4e3d-8deb-4d15ca9034df admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:06.918570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d48034-cced-4e3d-8deb-4d15ca9034df admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:06.927582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d469f-fac4-4b03-8c43-b1eba9607262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:06.937781 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d469f-fac4-4b03-8c43-b1eba9607262 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:06.937781 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d469f-fac4-4b03-8c43-b1eba9607262 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:06.937781 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d469f-fac4-4b03-8c43-b1eba9607262 admin admin] PATH_INFO: `/v3/projects/328f9e9600ca45098513424016a15eb3/users/a554958a6dff4169927a0ced159596e4/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:06.937781 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d469f-fac4-4b03-8c43-b1eba9607262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=328f9e9600ca45098513424016a15eb3, user_id=a554958a6dff4169927a0ced159596e4, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:06.942782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d469f-fac4-4b03-8c43-b1eba9607262 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:06.999034 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 957/1911] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:06 2022] PUT /identity/v3/projects/328f9e9600ca45098513424016a15eb3/users/a554958a6dff4169927a0ced159596e4/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:07.021726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c98232-5dd2-48f5-8fd0-54fb8f5e0510 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:07.023715 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c98232-5dd2-48f5-8fd0-54fb8f5e0510 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:07.024297 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c98232-5dd2-48f5-8fd0-54fb8f5e0510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:07.038195 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:07.053305 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2e2a9-8e1d-4b06-b5a6-b37b002af0b1 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:07.099886 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-c3c98232-5dd2-48f5-8fd0-54fb8f5e0510 None None] MFA Rules not processed for user `a554958a6dff4169927a0ced159596e4`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:07.101574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c3c98232-5dd2-48f5-8fd0-54fb8f5e0510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:07.174398 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc81a3-0861-4564-bc18-740c0be47b20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:07.179208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc81a3-0861-4564-bc18-740c0be47b20 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:07.180128 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc81a3-0861-4564-bc18-740c0be47b20 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:07.180770 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc81a3-0861-4564-bc18-740c0be47b20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:07.290457 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:07.311803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 958/1912] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:34:07.331681 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c98232-5dd2-48f5-8fd0-54fb8f5e0510 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:07.334157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c3c98232-5dd2-48f5-8fd0-54fb8f5e0510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:07.351473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc81a3-0861-4564-bc18-740c0be47b20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:07.354074 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc81a3-0861-4564-bc18-740c0be47b20 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:07.359767 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c98232-5dd2-48f5-8fd0-54fb8f5e0510 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:07.371415 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 955/1913] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:34:07.373733 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70dfea-03a1-447d-8708-780edf786bac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:07.374497 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70dfea-03a1-447d-8708-780edf786bac service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:07.375347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70dfea-03a1-447d-8708-780edf786bac service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:07.375703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70dfea-03a1-447d-8708-780edf786bac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:07.402105 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70dfea-03a1-447d-8708-780edf786bac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:07.402341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc81a3-0861-4564-bc18-740c0be47b20 service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:07.403932 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e70dfea-03a1-447d-8708-780edf786bac service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:07.419099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 959/1914] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:07.520020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc81a3-0861-4564-bc18-740c0be47b20 service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:07.527532 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ebf33-9f99-4b6e-8644-efc875e910b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:07.529523 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ebf33-9f99-4b6e-8644-efc875e910b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:07.529946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ebf33-9f99-4b6e-8644-efc875e910b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:07.530359 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ebf33-9f99-4b6e-8644-efc875e910b0 admin admin] PATH_INFO: `/v3/users/d2104f97b9cc41d2a61d34e64a5d19f9` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:07.532923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ebf33-9f99-4b6e-8644-efc875e910b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2104f97b9cc41d2a61d34e64a5d19f9)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:07.536915 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ebf33-9f99-4b6e-8644-efc875e910b0 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:07.927264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9c1ebf33-9f99-4b6e-8644-efc875e910b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2104f97b9cc41d2a61d34e64a5d19f9'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:07.955164 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9c1ebf33-9f99-4b6e-8644-efc875e910b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2104f97b9cc41d2a61d34e64a5d19f9'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:07.955164 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9c1ebf33-9f99-4b6e-8644-efc875e910b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2104f97b9cc41d2a61d34e64a5d19f9'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:07.966607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9c1ebf33-9f99-4b6e-8644-efc875e910b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2104f97b9cc41d2a61d34e64a5d19f9'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:07.967967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9c1ebf33-9f99-4b6e-8644-efc875e910b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2104f97b9cc41d2a61d34e64a5d19f9'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:08.071078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 956/1915] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:07 2022] DELETE /identity/v3/users/d2104f97b9cc41d2a61d34e64a5d19f9 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:08.077072 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a0c30-1e2e-4ba7-ae0f-b9ca033c008c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:08.077921 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a0c30-1e2e-4ba7-ae0f-b9ca033c008c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:08.078071 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a0c30-1e2e-4ba7-ae0f-b9ca033c008c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:08.123041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-a33a0c30-1e2e-4ba7-ae0f-b9ca033c008c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:08.125452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a33a0c30-1e2e-4ba7-ae0f-b9ca033c008c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:08.251018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 960/1916] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:08.271524 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ebf33-9f99-4b6e-8644-efc875e910b0 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:08.273110 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a0c30-1e2e-4ba7-ae0f-b9ca033c008c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:08.273291 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9c1ebf33-9f99-4b6e-8644-efc875e910b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:08.281360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a33a0c30-1e2e-4ba7-ae0f-b9ca033c008c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:08.309563 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a0c30-1e2e-4ba7-ae0f-b9ca033c008c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:08.315410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964e634-bba9-4701-8d14-30218fd32a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:08.316105 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964e634-bba9-4701-8d14-30218fd32a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:08.316382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964e634-bba9-4701-8d14-30218fd32a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:08.316701 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964e634-bba9-4701-8d14-30218fd32a6b admin admin] PATH_INFO: `/v3/users/d6140074c2014625b1b1f5402bea9457` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:08.317981 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964e634-bba9-4701-8d14-30218fd32a6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6140074c2014625b1b1f5402bea9457)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:08.320320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964e634-bba9-4701-8d14-30218fd32a6b admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:08.376390 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ebf33-9f99-4b6e-8644-efc875e910b0 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:08.382908 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93044ae0-6eda-4e54-b260-a3031ba06b3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:08.384099 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93044ae0-6eda-4e54-b260-a3031ba06b3e service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:08.385141 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93044ae0-6eda-4e54-b260-a3031ba06b3e service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:08.385644 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93044ae0-6eda-4e54-b260-a3031ba06b3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:08.405598 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93044ae0-6eda-4e54-b260-a3031ba06b3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:08.408613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93044ae0-6eda-4e54-b260-a3031ba06b3e service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:08.430046 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 957/1917] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:34:08.570695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-3964e634-bba9-4701-8d14-30218fd32a6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6140074c2014625b1b1f5402bea9457'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:08.643659 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-3964e634-bba9-4701-8d14-30218fd32a6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6140074c2014625b1b1f5402bea9457'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:08.644595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-3964e634-bba9-4701-8d14-30218fd32a6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6140074c2014625b1b1f5402bea9457'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:08.645460 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-3964e634-bba9-4701-8d14-30218fd32a6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6140074c2014625b1b1f5402bea9457'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:08.661129 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-3964e634-bba9-4701-8d14-30218fd32a6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6140074c2014625b1b1f5402bea9457'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:08.693250 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 961/1918] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:08 2022] DELETE /identity/v3/users/d6140074c2014625b1b1f5402bea9457 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:08.699618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b5c06-d630-4bfa-8040-36cc2b53f288 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:08.700550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b5c06-d630-4bfa-8040-36cc2b53f288 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:08.700970 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b5c06-d630-4bfa-8040-36cc2b53f288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:08.750268 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-d02b5c06-d630-4bfa-8040-36cc2b53f288 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:08.751613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d02b5c06-d630-4bfa-8040-36cc2b53f288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:08.872489 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 958/1919] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:34:08.884092 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964e634-bba9-4701-8d14-30218fd32a6b admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:08.887469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-3964e634-bba9-4701-8d14-30218fd32a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:08.976904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964e634-bba9-4701-8d14-30218fd32a6b admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:08.982216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c86537-17c9-4bb0-998b-6319d36f0282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:08.982874 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c86537-17c9-4bb0-998b-6319d36f0282 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:08.983171 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c86537-17c9-4bb0-998b-6319d36f0282 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:08.983446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c86537-17c9-4bb0-998b-6319d36f0282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:08.994724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c86537-17c9-4bb0-998b-6319d36f0282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:08.996095 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c86537-17c9-4bb0-998b-6319d36f0282 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:09.006876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 962/1920] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:09.075266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b5c06-d630-4bfa-8040-36cc2b53f288 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:09.077676 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:09.078496 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d02b5c06-d630-4bfa-8040-36cc2b53f288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:09.102012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b5c06-d630-4bfa-8040-36cc2b53f288 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:09.109825 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88587541-a9ef-4f5c-a7b1-63a19bf8cda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:09.111223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88587541-a9ef-4f5c-a7b1-63a19bf8cda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:09.111821 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88587541-a9ef-4f5c-a7b1-63a19bf8cda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:09.112385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88587541-a9ef-4f5c-a7b1-63a19bf8cda1 admin admin] PATH_INFO: `/v3/projects/b0d492f6eb8549978df1ff49b8b5a186` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:09.114649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88587541-a9ef-4f5c-a7b1-63a19bf8cda1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0d492f6eb8549978df1ff49b8b5a186)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:09.118006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88587541-a9ef-4f5c-a7b1-63a19bf8cda1 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:09.331758 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-88587541-a9ef-4f5c-a7b1-63a19bf8cda1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0d492f6eb8549978df1ff49b8b5a186', 'request_id': 'req-88587541-a9ef-4f5c-a7b1-63a19bf8cda1'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:09.347374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-88587541-a9ef-4f5c-a7b1-63a19bf8cda1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0d492f6eb8549978df1ff49b8b5a186', 'request_id': 'req-88587541-a9ef-4f5c-a7b1-63a19bf8cda1'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:09.455182 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-88587541-a9ef-4f5c-a7b1-63a19bf8cda1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0d492f6eb8549978df1ff49b8b5a186', 'request_id': 'req-88587541-a9ef-4f5c-a7b1-63a19bf8cda1'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:09.486967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:09.501585 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-88587541-a9ef-4f5c-a7b1-63a19bf8cda1 admin admin] The token cache is being invalidate because project b0d492f6eb8549978df1ff49b8b5a186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:09.502247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-88587541-a9ef-4f5c-a7b1-63a19bf8cda1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:09.503460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-88587541-a9ef-4f5c-a7b1-63a19bf8cda1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:09.505748 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 959/1921] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:09 2022] DELETE /identity/v3/projects/b0d492f6eb8549978df1ff49b8b5a186 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:09.513217 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cce389-c09b-49e4-b82b-080295a6cd8c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:09.513521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cce389-c09b-49e4-b82b-080295a6cd8c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:09.513759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cce389-c09b-49e4-b82b-080295a6cd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:09.537995 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b0cce389-c09b-49e4-b82b-080295a6cd8c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:09.539390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b0cce389-c09b-49e4-b82b-080295a6cd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:09.651774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 963/1922] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:09.660627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac8ce3-4133-4ab9-b212-fb05a0291e22 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:09.660813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac8ce3-4133-4ab9-b212-fb05a0291e22 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:09.661092 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac8ce3-4133-4ab9-b212-fb05a0291e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:09.685109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-48ac8ce3-4133-4ab9-b212-fb05a0291e22 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:09.686368 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-48ac8ce3-4133-4ab9-b212-fb05a0291e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:09.696192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 960/1923] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:09.701974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cce389-c09b-49e4-b82b-080295a6cd8c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:09.706126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac8ce3-4133-4ab9-b212-fb05a0291e22 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:09.708236 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-48ac8ce3-4133-4ab9-b212-fb05a0291e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:09.718439 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cce389-c09b-49e4-b82b-080295a6cd8c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:09.721786 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac8ce3-4133-4ab9-b212-fb05a0291e22 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:09.725692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2430d6-a986-46c3-844b-9d99c70f9e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:09.727175 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2430d6-a986-46c3-844b-9d99c70f9e87 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:09.727628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2430d6-a986-46c3-844b-9d99c70f9e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:09.728061 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2430d6-a986-46c3-844b-9d99c70f9e87 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:09.728780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99e385-1747-4b9e-a396-e0254ea26d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:09.729534 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99e385-1747-4b9e-a396-e0254ea26d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:09.729885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99e385-1747-4b9e-a396-e0254ea26d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:09.730178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99e385-1747-4b9e-a396-e0254ea26d33 admin admin] PATH_INFO: `/v3/projects/e1b80031cca5465695bd0724b37c9fe3` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:09.731288 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2430d6-a986-46c3-844b-9d99c70f9e87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:34:09.731760 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99e385-1747-4b9e-a396-e0254ea26d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1b80031cca5465695bd0724b37c9fe3)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:09.731918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2430d6-a986-46c3-844b-9d99c70f9e87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:09.734151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99e385-1747-4b9e-a396-e0254ea26d33 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:09.734394 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2430d6-a986-46c3-844b-9d99c70f9e87 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:09.757713 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 964/1924] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:34:09.769556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2430d6-a986-46c3-844b-9d99c70f9e87 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:09.784734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2430d6-a986-46c3-844b-9d99c70f9e87 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:09.792322 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618425e-d812-4d0c-b0fc-86f1a4a53d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:09.793400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618425e-d812-4d0c-b0fc-86f1a4a53d3c admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:09.804203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618425e-d812-4d0c-b0fc-86f1a4a53d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:09.804203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618425e-d812-4d0c-b0fc-86f1a4a53d3c admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:09.804203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618425e-d812-4d0c-b0fc-86f1a4a53d3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:09.804203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618425e-d812-4d0c-b0fc-86f1a4a53d3c admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:09.908969 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2e99e385-1747-4b9e-a396-e0254ea26d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1b80031cca5465695bd0724b37c9fe3', 'request_id': 'req-2e99e385-1747-4b9e-a396-e0254ea26d33'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:09.921092 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2e99e385-1747-4b9e-a396-e0254ea26d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1b80031cca5465695bd0724b37c9fe3', 'request_id': 'req-2e99e385-1747-4b9e-a396-e0254ea26d33'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:09.977839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 965/1925] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:09 2022] POST /identity/v3/projects => generated 385 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:34:09.983358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618425e-d812-4d0c-b0fc-86f1a4a53d3c admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:10.001298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2e99e385-1747-4b9e-a396-e0254ea26d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1b80031cca5465695bd0724b37c9fe3', 'request_id': 'req-2e99e385-1747-4b9e-a396-e0254ea26d33'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:10.014388 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2e99e385-1747-4b9e-a396-e0254ea26d33 admin admin] The token cache is being invalidate because project e1b80031cca5465695bd0724b37c9fe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:10.014679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2e99e385-1747-4b9e-a396-e0254ea26d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:10.015439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2e99e385-1747-4b9e-a396-e0254ea26d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:10.017014 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 961/1926] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:09 2022] DELETE /identity/v3/projects/e1b80031cca5465695bd0724b37c9fe3 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:10.021622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99e385-1747-4b9e-a396-e0254ea26d33 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:10.023159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2e99e385-1747-4b9e-a396-e0254ea26d33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:10.083835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618425e-d812-4d0c-b0fc-86f1a4a53d3c admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:10.092020 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772eddf2-38ff-415e-96dd-4952ccb08676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:10.092985 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772eddf2-38ff-415e-96dd-4952ccb08676 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:10.093458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772eddf2-38ff-415e-96dd-4952ccb08676 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:10.093925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772eddf2-38ff-415e-96dd-4952ccb08676 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:10.095042 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772eddf2-38ff-415e-96dd-4952ccb08676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:10.098120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772eddf2-38ff-415e-96dd-4952ccb08676 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:10.110342 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99e385-1747-4b9e-a396-e0254ea26d33 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:10.115393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4518118-4748-47ab-8971-944d36af12ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:10.116055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4518118-4748-47ab-8971-944d36af12ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:10.116338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4518118-4748-47ab-8971-944d36af12ef admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:10.116614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4518118-4748-47ab-8971-944d36af12ef admin admin] PATH_INFO: `/v3/users/a554958a6dff4169927a0ced159596e4` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:10.117874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4518118-4748-47ab-8971-944d36af12ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=a554958a6dff4169927a0ced159596e4)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:10.120940 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4518118-4748-47ab-8971-944d36af12ef admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:10.264347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 966/1927] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:09 2022] POST /identity/v3/users => generated 293 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:34:10.269823 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772eddf2-38ff-415e-96dd-4952ccb08676 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:10.271730 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-772eddf2-38ff-415e-96dd-4952ccb08676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:10.285835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772eddf2-38ff-415e-96dd-4952ccb08676 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:10.292185 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ceeeec-d96f-435f-a3ea-a5e964e3eb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:10.292822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ceeeec-d96f-435f-a3ea-a5e964e3eb7e admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:10.293111 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ceeeec-d96f-435f-a3ea-a5e964e3eb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:10.293409 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ceeeec-d96f-435f-a3ea-a5e964e3eb7e admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:10.294057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ceeeec-d96f-435f-a3ea-a5e964e3eb7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:10.296598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ceeeec-d96f-435f-a3ea-a5e964e3eb7e admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:10.316876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 967/1928] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:10.321781 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ceeeec-d96f-435f-a3ea-a5e964e3eb7e admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:10.333636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ceeeec-d96f-435f-a3ea-a5e964e3eb7e admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:10.340621 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c6539-275d-4052-b9d8-ffb6c27164ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:10.343755 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c6539-275d-4052-b9d8-ffb6c27164ea admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:10.343755 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c6539-275d-4052-b9d8-ffb6c27164ea admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:10.343755 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c6539-275d-4052-b9d8-ffb6c27164ea admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:10.344144 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c6539-275d-4052-b9d8-ffb6c27164ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:10.346932 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c6539-275d-4052-b9d8-ffb6c27164ea admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:10.368835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 968/1929] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:34:10.373909 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c6539-275d-4052-b9d8-ffb6c27164ea admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:10.387360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c6539-275d-4052-b9d8-ffb6c27164ea admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:10.394890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d0f69-34d9-4e8f-9a73-fb3f4d75c850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:10.395929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d0f69-34d9-4e8f-9a73-fb3f4d75c850 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:10.396251 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d0f69-34d9-4e8f-9a73-fb3f4d75c850 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:10.396561 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d0f69-34d9-4e8f-9a73-fb3f4d75c850 admin admin] PATH_INFO: `/v3/projects/03b3c274843b4914aca54164afe46a34/users/0241c434146544039f23d3da384779f6/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:10.416476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d0f69-34d9-4e8f-9a73-fb3f4d75c850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03b3c274843b4914aca54164afe46a34, user_id=0241c434146544039f23d3da384779f6, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:10.420597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d0f69-34d9-4e8f-9a73-fb3f4d75c850 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:10.449016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e4518118-4748-47ab-8971-944d36af12ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a554958a6dff4169927a0ced159596e4'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:10.463653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e4518118-4748-47ab-8971-944d36af12ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a554958a6dff4169927a0ced159596e4'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:10.464762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e4518118-4748-47ab-8971-944d36af12ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a554958a6dff4169927a0ced159596e4'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:10.477760 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e4518118-4748-47ab-8971-944d36af12ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a554958a6dff4169927a0ced159596e4'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:10.479285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e4518118-4748-47ab-8971-944d36af12ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a554958a6dff4169927a0ced159596e4'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:10.521219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 969/1930] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:10 2022] PUT /identity/v3/projects/03b3c274843b4914aca54164afe46a34/users/0241c434146544039f23d3da384779f6/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:10.536492 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fce9d8-1d03-4d2f-82ed-bac380e3d251 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:10.536833 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fce9d8-1d03-4d2f-82ed-bac380e3d251 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:10.537333 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fce9d8-1d03-4d2f-82ed-bac380e3d251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:10.575308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-e5fce9d8-1d03-4d2f-82ed-bac380e3d251 None None] MFA Rules not processed for user `0241c434146544039f23d3da384779f6`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:10.576831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e5fce9d8-1d03-4d2f-82ed-bac380e3d251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:10.630657 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 962/1931] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:10 2022] DELETE /identity/v3/users/a554958a6dff4169927a0ced159596e4 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:10.701518 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 970/1932] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:10.707395 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4518118-4748-47ab-8971-944d36af12ef admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:10.709785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e4518118-4748-47ab-8971-944d36af12ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:10.807330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4518118-4748-47ab-8971-944d36af12ef admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:10.817760 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d294e5-fb5b-4193-9f63-5b847bfb3f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:10.818677 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d294e5-fb5b-4193-9f63-5b847bfb3f87 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:10.819134 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d294e5-fb5b-4193-9f63-5b847bfb3f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:10.819572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d294e5-fb5b-4193-9f63-5b847bfb3f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:10.820491 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d294e5-fb5b-4193-9f63-5b847bfb3f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:10.824121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d294e5-fb5b-4193-9f63-5b847bfb3f87 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:10.982099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:10.994609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 963/1933] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:10 2022] POST /identity/v3/projects => generated 383 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:11.001446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fce9d8-1d03-4d2f-82ed-bac380e3d251 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:11.119795 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fce9d8-1d03-4d2f-82ed-bac380e3d251 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:11.208261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b72f0-b922-44b2-bf61-37cfea14d2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:11.209433 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b72f0-b922-44b2-bf61-37cfea14d2c2 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:11.209916 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b72f0-b922-44b2-bf61-37cfea14d2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:11.210437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b72f0-b922-44b2-bf61-37cfea14d2c2 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:11.211450 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b72f0-b922-44b2-bf61-37cfea14d2c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:11.215189 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b72f0-b922-44b2-bf61-37cfea14d2c2 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:11.349056 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 971/1934] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:10 2022] POST /identity/v3/users => generated 292 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:34:11.356338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d294e5-fb5b-4193-9f63-5b847bfb3f87 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:11.379413 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d294e5-fb5b-4193-9f63-5b847bfb3f87 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:11.384568 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b72f0-b922-44b2-bf61-37cfea14d2c2 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:11.388714 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632192a4-08e6-4180-9df4-65d8bce31d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:11.389119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632192a4-08e6-4180-9df4-65d8bce31d87 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:11.389745 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632192a4-08e6-4180-9df4-65d8bce31d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:11.390325 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632192a4-08e6-4180-9df4-65d8bce31d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:11.391332 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632192a4-08e6-4180-9df4-65d8bce31d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:11.395387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632192a4-08e6-4180-9df4-65d8bce31d87 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:11.401553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:11.401870 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-275b72f0-b922-44b2-bf61-37cfea14d2c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:11.431995 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 964/1935] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:11.432387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b72f0-b922-44b2-bf61-37cfea14d2c2 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:11.436323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632192a4-08e6-4180-9df4-65d8bce31d87 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:11.446920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12542062-6ba6-41e4-b66b-d096811bdafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:11.446920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12542062-6ba6-41e4-b66b-d096811bdafe admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:11.446920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12542062-6ba6-41e4-b66b-d096811bdafe admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:11.446920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12542062-6ba6-41e4-b66b-d096811bdafe admin admin] PATH_INFO: `/v3/projects/328f9e9600ca45098513424016a15eb3` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:11.447524 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12542062-6ba6-41e4-b66b-d096811bdafe admin admin] RBAC: Authorizing `identity:delete_project(project_id=328f9e9600ca45098513424016a15eb3)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:11.447524 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12542062-6ba6-41e4-b66b-d096811bdafe admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:11.459298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632192a4-08e6-4180-9df4-65d8bce31d87 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:11.466565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b60b4c-790f-4b89-85d2-a201d47a2447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:11.467821 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b60b4c-790f-4b89-85d2-a201d47a2447 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:11.468326 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b60b4c-790f-4b89-85d2-a201d47a2447 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:11.468827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b60b4c-790f-4b89-85d2-a201d47a2447 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:11.470163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b60b4c-790f-4b89-85d2-a201d47a2447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:11.473872 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b60b4c-790f-4b89-85d2-a201d47a2447 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:11.495265 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 965/1936] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:11.496434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b60b4c-790f-4b89-85d2-a201d47a2447 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:11.511865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b60b4c-790f-4b89-85d2-a201d47a2447 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:11.521218 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da2757-66a9-4d09-a551-314885c1898d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:11.522426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da2757-66a9-4d09-a551-314885c1898d admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:11.522919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da2757-66a9-4d09-a551-314885c1898d admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:11.523427 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da2757-66a9-4d09-a551-314885c1898d admin admin] PATH_INFO: `/v3/projects/786597f70c234b66a329c99876ea2304/users/85d420ad40a44642b6acf4edf9791e65/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:11.560279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da2757-66a9-4d09-a551-314885c1898d admin admin] RBAC: Authorizing `identity:create_grant(project_id=786597f70c234b66a329c99876ea2304, user_id=85d420ad40a44642b6acf4edf9791e65, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:11.564845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da2757-66a9-4d09-a551-314885c1898d admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:11.758939 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 966/1937] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:11 2022] PUT /identity/v3/projects/786597f70c234b66a329c99876ea2304/users/85d420ad40a44642b6acf4edf9791e65/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:11.770842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b71e08-be61-4326-9df5-85082518cc30 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:11.771541 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b71e08-be61-4326-9df5-85082518cc30 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:11.772036 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b71e08-be61-4326-9df5-85082518cc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:11.781236 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-12542062-6ba6-41e4-b66b-d096811bdafe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '328f9e9600ca45098513424016a15eb3', 'request_id': 'req-12542062-6ba6-41e4-b66b-d096811bdafe'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:11.795471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-12542062-6ba6-41e4-b66b-d096811bdafe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '328f9e9600ca45098513424016a15eb3', 'request_id': 'req-12542062-6ba6-41e4-b66b-d096811bdafe'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:11.826504 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-15b71e08-be61-4326-9df5-85082518cc30 None None] MFA Rules not processed for user `85d420ad40a44642b6acf4edf9791e65`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:11.827836 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-15b71e08-be61-4326-9df5-85082518cc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:11.933305 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 967/1938] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:34:11.938035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b71e08-be61-4326-9df5-85082518cc30 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:11.942905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-12542062-6ba6-41e4-b66b-d096811bdafe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '328f9e9600ca45098513424016a15eb3', 'request_id': 'req-12542062-6ba6-41e4-b66b-d096811bdafe'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:11.967486 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-12542062-6ba6-41e4-b66b-d096811bdafe admin admin] The token cache is being invalidate because project 328f9e9600ca45098513424016a15eb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:11.967961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-12542062-6ba6-41e4-b66b-d096811bdafe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:11.968998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-12542062-6ba6-41e4-b66b-d096811bdafe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:11.975447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 972/1939] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:11 2022] DELETE /identity/v3/projects/328f9e9600ca45098513424016a15eb3 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:11.977630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2329189-eeb7-46be-b305-fea5e59f5f04 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:11.977911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2329189-eeb7-46be-b305-fea5e59f5f04 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:11.979192 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2329189-eeb7-46be-b305-fea5e59f5f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:12.010704 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b2329189-eeb7-46be-b305-fea5e59f5f04 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:12.012569 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b2329189-eeb7-46be-b305-fea5e59f5f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:12.079732 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b71e08-be61-4326-9df5-85082518cc30 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:12.093125 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4d51a-219b-4292-8325-d07e0c4b8132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:12.093819 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4d51a-219b-4292-8325-d07e0c4b8132 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:12.094216 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4d51a-219b-4292-8325-d07e0c4b8132 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:12.094461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4d51a-219b-4292-8325-d07e0c4b8132 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:12.095457 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4d51a-219b-4292-8325-d07e0c4b8132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:12.097495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4d51a-219b-4292-8325-d07e0c4b8132 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:12.143645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 973/1940] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:12.151513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b1f71-4a6b-4083-9aa0-0362ea354075 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:12.151783 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b1f71-4a6b-4083-9aa0-0362ea354075 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:12.152059 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b1f71-4a6b-4083-9aa0-0362ea354075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:12.178799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-897b1f71-4a6b-4083-9aa0-0362ea354075 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:12.180158 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-897b1f71-4a6b-4083-9aa0-0362ea354075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:12.194047 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 974/1941] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:12.207261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b1f71-4a6b-4083-9aa0-0362ea354075 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:12.212550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 968/1942] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:11 2022] POST /identity/v3/projects => generated 385 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:34:12.235688 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4d51a-219b-4292-8325-d07e0c4b8132 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:12.235688 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-cbf4d51a-219b-4292-8325-d07e0c4b8132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:12.250927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b1f71-4a6b-4083-9aa0-0362ea354075 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:12.352785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4d51a-219b-4292-8325-d07e0c4b8132 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:12.355904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78f819-3eaf-4a9a-a385-474ba754373b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:12.356990 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78f819-3eaf-4a9a-a385-474ba754373b admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:12.357429 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78f819-3eaf-4a9a-a385-474ba754373b admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:12.357844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78f819-3eaf-4a9a-a385-474ba754373b admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:12.358687 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78f819-3eaf-4a9a-a385-474ba754373b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:34:12.359252 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78f819-3eaf-4a9a-a385-474ba754373b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:12.359637 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58695d1b-9562-410a-98c5-1b3beb65a0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:12.360338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58695d1b-9562-410a-98c5-1b3beb65a0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:12.360632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58695d1b-9562-410a-98c5-1b3beb65a0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:12.360910 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58695d1b-9562-410a-98c5-1b3beb65a0e8 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:12.361635 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58695d1b-9562-410a-98c5-1b3beb65a0e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:12.362326 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78f819-3eaf-4a9a-a385-474ba754373b admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:12.364775 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58695d1b-9562-410a-98c5-1b3beb65a0e8 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:12.383795 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 975/1943] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:34:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:34:12.385045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78f819-3eaf-4a9a-a385-474ba754373b admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:12.396059 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78f819-3eaf-4a9a-a385-474ba754373b admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:12.404814 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c2c3e-7806-4114-b585-d4d2eed97b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:12.405450 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c2c3e-7806-4114-b585-d4d2eed97b08 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:12.405739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c2c3e-7806-4114-b585-d4d2eed97b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:12.406053 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c2c3e-7806-4114-b585-d4d2eed97b08 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:12.407291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c2c3e-7806-4114-b585-d4d2eed97b08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:12.409469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c2c3e-7806-4114-b585-d4d2eed97b08 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:12.498491 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 969/1944] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:12 2022] POST /identity/v3/users => generated 299 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:12.502614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58695d1b-9562-410a-98c5-1b3beb65a0e8 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:12.514282 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58695d1b-9562-410a-98c5-1b3beb65a0e8 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:12.535729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 976/1945] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:12 2022] POST /identity/v3/projects => generated 359 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:34:12.538744 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0e6ac-773b-44f9-b0a5-ab49e9cf5f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:12.538744 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0e6ac-773b-44f9-b0a5-ab49e9cf5f47 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:12.538744 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0e6ac-773b-44f9-b0a5-ab49e9cf5f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:12.538744 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0e6ac-773b-44f9-b0a5-ab49e9cf5f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:12.540140 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0e6ac-773b-44f9-b0a5-ab49e9cf5f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:12.542400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c2c3e-7806-4114-b585-d4d2eed97b08 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:12.543929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0e6ac-773b-44f9-b0a5-ab49e9cf5f47 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:12.570825 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 970/1946] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:34:12.579552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0e6ac-773b-44f9-b0a5-ab49e9cf5f47 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:12.593798 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0e6ac-773b-44f9-b0a5-ab49e9cf5f47 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:12.677036 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c2c3e-7806-4114-b585-d4d2eed97b08 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:12.693532 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e925d5-e0e8-454c-aa63-b5d56d0c0ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:12.694636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e925d5-e0e8-454c-aa63-b5d56d0c0ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:12.695958 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e925d5-e0e8-454c-aa63-b5d56d0c0ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:12.696480 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e925d5-e0e8-454c-aa63-b5d56d0c0ed4 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:12.698695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e925d5-e0e8-454c-aa63-b5d56d0c0ed4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:12.701016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e925d5-e0e8-454c-aa63-b5d56d0c0ed4 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:12.706281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e82dd-2c41-43d5-ae64-ef4540013956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:12.707574 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e82dd-2c41-43d5-ae64-ef4540013956 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:12.708068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e82dd-2c41-43d5-ae64-ef4540013956 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:12.708896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e82dd-2c41-43d5-ae64-ef4540013956 admin admin] PATH_INFO: `/v3/projects/985f59ae81d046a3986d7896fdd3ba1a/users/1636d7db79ae4c04890bc8b2630647d6/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:12.745316 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e82dd-2c41-43d5-ae64-ef4540013956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=985f59ae81d046a3986d7896fdd3ba1a, user_id=1636d7db79ae4c04890bc8b2630647d6, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:12.750212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e82dd-2c41-43d5-ae64-ef4540013956 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:12.865613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 971/1947] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:12 2022] PUT /identity/v3/projects/985f59ae81d046a3986d7896fdd3ba1a/users/1636d7db79ae4c04890bc8b2630647d6/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:12.867009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e82dd-2c41-43d5-ae64-ef4540013956 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:12.874528 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 977/1948] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:12 2022] POST /identity/v3/users => generated 280 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:12.886090 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e925d5-e0e8-454c-aa63-b5d56d0c0ed4 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:12.900351 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e925d5-e0e8-454c-aa63-b5d56d0c0ed4 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:12.989660 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e82dd-2c41-43d5-ae64-ef4540013956 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:12.993671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f1ec8-1ec7-4a97-95d8-c07bd8b8201f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:13.001321 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f1ec8-1ec7-4a97-95d8-c07bd8b8201f admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:13.001321 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f1ec8-1ec7-4a97-95d8-c07bd8b8201f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:13.001630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c9d06-f149-45ab-bfe5-959b349dffda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:13.001806 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f1ec8-1ec7-4a97-95d8-c07bd8b8201f admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:13.001984 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c9d06-f149-45ab-bfe5-959b349dffda admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:13.001984 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c9d06-f149-45ab-bfe5-959b349dffda admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:13.002238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f1ec8-1ec7-4a97-95d8-c07bd8b8201f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:13.002366 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c9d06-f149-45ab-bfe5-959b349dffda admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:13.002366 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c9d06-f149-45ab-bfe5-959b349dffda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:13.002601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f1ec8-1ec7-4a97-95d8-c07bd8b8201f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:13.002738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c9d06-f149-45ab-bfe5-959b349dffda admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:13.008557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 978/1949] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:13.013579 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f1ec8-1ec7-4a97-95d8-c07bd8b8201f admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:13.014087 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 972/1950] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:34:13.019385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c9d06-f149-45ab-bfe5-959b349dffda admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:13.026125 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f1ec8-1ec7-4a97-95d8-c07bd8b8201f admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:13.039453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c9d06-f149-45ab-bfe5-959b349dffda admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:13.045380 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1f35b-d904-4eb5-9312-a99c3c4d469d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:13.046186 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1f35b-d904-4eb5-9312-a99c3c4d469d admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:13.046393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed6c5f-d640-4588-a4b2-60ddd5397188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:13.046572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1f35b-d904-4eb5-9312-a99c3c4d469d admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:13.046707 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1f35b-d904-4eb5-9312-a99c3c4d469d admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:13.047818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed6c5f-d640-4588-a4b2-60ddd5397188 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:13.047818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed6c5f-d640-4588-a4b2-60ddd5397188 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:13.048132 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1f35b-d904-4eb5-9312-a99c3c4d469d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:13.048264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed6c5f-d640-4588-a4b2-60ddd5397188 admin admin] PATH_INFO: `/v3/projects/985f59ae81d046a3986d7896fdd3ba1a/users/1636d7db79ae4c04890bc8b2630647d6/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:13.049491 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1f35b-d904-4eb5-9312-a99c3c4d469d admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:13.050079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed6c5f-d640-4588-a4b2-60ddd5397188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=985f59ae81d046a3986d7896fdd3ba1a, user_id=1636d7db79ae4c04890bc8b2630647d6, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:13.053239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed6c5f-d640-4588-a4b2-60ddd5397188 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:13.062502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 979/1951] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:34:13.067827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 973/1952] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:13 2022] PUT /identity/v3/projects/985f59ae81d046a3986d7896fdd3ba1a/users/1636d7db79ae4c04890bc8b2630647d6/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:13.086169 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1f35b-d904-4eb5-9312-a99c3c4d469d admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:13.086659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645988d4-7214-4d56-9a9f-6b924ef69fdc None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:13.086659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645988d4-7214-4d56-9a9f-6b924ef69fdc None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:13.086659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645988d4-7214-4d56-9a9f-6b924ef69fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:13.087054 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1f35b-d904-4eb5-9312-a99c3c4d469d admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:13.124144 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-645988d4-7214-4d56-9a9f-6b924ef69fdc None None] MFA Rules not processed for user `1636d7db79ae4c04890bc8b2630647d6`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:13.125419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-645988d4-7214-4d56-9a9f-6b924ef69fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:13.193525 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289534a-ea4b-44cf-ad70-070ec13f5164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:13.194456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289534a-ea4b-44cf-ad70-070ec13f5164 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:13.194904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289534a-ea4b-44cf-ad70-070ec13f5164 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:13.195286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289534a-ea4b-44cf-ad70-070ec13f5164 admin admin] PATH_INFO: `/v3/projects/51371556683046e8acfa107cadd0b6f8/users/b9d0a3134ddc401ea3f962078b418354/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:13.238918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289534a-ea4b-44cf-ad70-070ec13f5164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51371556683046e8acfa107cadd0b6f8, user_id=b9d0a3134ddc401ea3f962078b418354, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:13.247858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289534a-ea4b-44cf-ad70-070ec13f5164 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:13.257805 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:13.258110 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 974/1953] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:34:13.276607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645988d4-7214-4d56-9a9f-6b924ef69fdc None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:13.278113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-645988d4-7214-4d56-9a9f-6b924ef69fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:13.359513 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645988d4-7214-4d56-9a9f-6b924ef69fdc None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:13.365122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106abe85-de6d-40ac-a706-f947858f1606 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:13.365938 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106abe85-de6d-40ac-a706-f947858f1606 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:13.366191 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106abe85-de6d-40ac-a706-f947858f1606 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:13.366503 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106abe85-de6d-40ac-a706-f947858f1606 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:13.368695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-106abe85-de6d-40ac-a706-f947858f1606 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:13.396815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 980/1954] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:13 2022] PUT /identity/v3/projects/51371556683046e8acfa107cadd0b6f8/users/b9d0a3134ddc401ea3f962078b418354/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:13.412860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c1e42-27eb-4de7-900b-325c15fb90d9 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:13.413117 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c1e42-27eb-4de7-900b-325c15fb90d9 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:13.413394 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c1e42-27eb-4de7-900b-325c15fb90d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:13.443005 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:13.474720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-528c1e42-27eb-4de7-900b-325c15fb90d9 None None] MFA Rules not processed for user `b9d0a3134ddc401ea3f962078b418354`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:13.481410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-528c1e42-27eb-4de7-900b-325c15fb90d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:13.506566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106abe85-de6d-40ac-a706-f947858f1606 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:13.510024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106abe85-de6d-40ac-a706-f947858f1606 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:13.530415 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 975/1955] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:13.583598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 981/1956] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:13 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:13.815344 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106abe85-de6d-40ac-a706-f947858f1606 service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:13.818242 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-106abe85-de6d-40ac-a706-f947858f1606 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:13.845233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106abe85-de6d-40ac-a706-f947858f1606 service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:13.940377 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf0827-a3fa-4396-9ba0-b08a878022c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:13.941536 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf0827-a3fa-4396-9ba0-b08a878022c5 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:13.942052 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf0827-a3fa-4396-9ba0-b08a878022c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:13.943297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf0827-a3fa-4396-9ba0-b08a878022c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:13.959681 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf0827-a3fa-4396-9ba0-b08a878022c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:13.963402 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf0827-a3fa-4396-9ba0-b08a878022c5 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:13.983027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 976/1957] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:13 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:14.000425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c1e42-27eb-4de7-900b-325c15fb90d9 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:14.115161 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c1e42-27eb-4de7-900b-325c15fb90d9 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:14.126361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91032a85-b125-4945-8ad9-f38fc4e4c801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:14.126361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91032a85-b125-4945-8ad9-f38fc4e4c801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:14.126361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91032a85-b125-4945-8ad9-f38fc4e4c801 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:14.126361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91032a85-b125-4945-8ad9-f38fc4e4c801 admin admin] PATH_INFO: `/v3/users/b9d0a3134ddc401ea3f962078b418354` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:14.126885 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91032a85-b125-4945-8ad9-f38fc4e4c801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9d0a3134ddc401ea3f962078b418354)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:14.127997 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91032a85-b125-4945-8ad9-f38fc4e4c801 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:14.372663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:14.502157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-91032a85-b125-4945-8ad9-f38fc4e4c801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9d0a3134ddc401ea3f962078b418354'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:14.680676 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-91032a85-b125-4945-8ad9-f38fc4e4c801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9d0a3134ddc401ea3f962078b418354'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:14.682106 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-91032a85-b125-4945-8ad9-f38fc4e4c801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9d0a3134ddc401ea3f962078b418354'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:14.683402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-91032a85-b125-4945-8ad9-f38fc4e4c801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9d0a3134ddc401ea3f962078b418354'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:14.693268 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-91032a85-b125-4945-8ad9-f38fc4e4c801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9d0a3134ddc401ea3f962078b418354'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:14.730595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 982/1958] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:13 2022] DELETE /identity/v3/users/b9d0a3134ddc401ea3f962078b418354 => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:14.737414 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217923f-be24-48c7-81f7-9f62ad938010 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:14.737900 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217923f-be24-48c7-81f7-9f62ad938010 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:14.738458 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217923f-be24-48c7-81f7-9f62ad938010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:14.780565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c217923f-be24-48c7-81f7-9f62ad938010 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:14.784819 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c217923f-be24-48c7-81f7-9f62ad938010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:14.918502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 977/1959] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:14.934803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91032a85-b125-4945-8ad9-f38fc4e4c801 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:14.936261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-91032a85-b125-4945-8ad9-f38fc4e4c801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:15.027646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91032a85-b125-4945-8ad9-f38fc4e4c801 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:15.032136 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb793c61-009f-4a05-9300-abb7e9e8feb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:15.032856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb793c61-009f-4a05-9300-abb7e9e8feb2 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:15.033180 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb793c61-009f-4a05-9300-abb7e9e8feb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:15.033500 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb793c61-009f-4a05-9300-abb7e9e8feb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:15.045674 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb793c61-009f-4a05-9300-abb7e9e8feb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:15.047327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb793c61-009f-4a05-9300-abb7e9e8feb2 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:15.059784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 983/1960] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:15.230109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:15.655651 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217923f-be24-48c7-81f7-9f62ad938010 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:15.657075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c217923f-be24-48c7-81f7-9f62ad938010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:15.683556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217923f-be24-48c7-81f7-9f62ad938010 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:15.690359 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f511bdf-1c87-4205-97b5-16c063a9e9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:15.691362 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f511bdf-1c87-4205-97b5-16c063a9e9d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:15.691722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f511bdf-1c87-4205-97b5-16c063a9e9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:15.692276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f511bdf-1c87-4205-97b5-16c063a9e9d9 admin admin] PATH_INFO: `/v3/projects/51371556683046e8acfa107cadd0b6f8` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:15.694055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f511bdf-1c87-4205-97b5-16c063a9e9d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51371556683046e8acfa107cadd0b6f8)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:15.704136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f511bdf-1c87-4205-97b5-16c063a9e9d9 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:15.876375 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5f511bdf-1c87-4205-97b5-16c063a9e9d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51371556683046e8acfa107cadd0b6f8', 'request_id': 'req-5f511bdf-1c87-4205-97b5-16c063a9e9d9'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:15.886700 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5f511bdf-1c87-4205-97b5-16c063a9e9d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51371556683046e8acfa107cadd0b6f8', 'request_id': 'req-5f511bdf-1c87-4205-97b5-16c063a9e9d9'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:15.955633 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5f511bdf-1c87-4205-97b5-16c063a9e9d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51371556683046e8acfa107cadd0b6f8', 'request_id': 'req-5f511bdf-1c87-4205-97b5-16c063a9e9d9'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:15.977392 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5f511bdf-1c87-4205-97b5-16c063a9e9d9 admin admin] The token cache is being invalidate because project 51371556683046e8acfa107cadd0b6f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:15.977809 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5f511bdf-1c87-4205-97b5-16c063a9e9d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:15.980128 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5f511bdf-1c87-4205-97b5-16c063a9e9d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:15.982152 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 978/1961] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:15 2022] DELETE /identity/v3/projects/51371556683046e8acfa107cadd0b6f8 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:15.990799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b692b2-6c8e-40c8-a76d-8de4103c68ee None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:15.991802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b692b2-6c8e-40c8-a76d-8de4103c68ee None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:15.991802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b692b2-6c8e-40c8-a76d-8de4103c68ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:16.020256 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-09b692b2-6c8e-40c8-a76d-8de4103c68ee None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:16.022292 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-09b692b2-6c8e-40c8-a76d-8de4103c68ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:16.121876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 984/1962] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:16.132578 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf89172-5ff1-46b9-a7e6-2cff0d64583c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:16.133084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf89172-5ff1-46b9-a7e6-2cff0d64583c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:16.133463 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf89172-5ff1-46b9-a7e6-2cff0d64583c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:16.162547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-acf89172-5ff1-46b9-a7e6-2cff0d64583c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:16.164413 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-acf89172-5ff1-46b9-a7e6-2cff0d64583c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:16.176119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 979/1963] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:16.182748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b692b2-6c8e-40c8-a76d-8de4103c68ee None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:16.198644 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b692b2-6c8e-40c8-a76d-8de4103c68ee None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:16.207056 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1243a-750e-41f1-9791-e6e9fa85575b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:16.208061 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1243a-750e-41f1-9791-e6e9fa85575b admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:16.208472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1243a-750e-41f1-9791-e6e9fa85575b admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:16.208896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1243a-750e-41f1-9791-e6e9fa85575b admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:16.209751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1243a-750e-41f1-9791-e6e9fa85575b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:34:16.210703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1243a-750e-41f1-9791-e6e9fa85575b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:16.213858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1243a-750e-41f1-9791-e6e9fa85575b admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:16.240174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 985/1964] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:34:16.248835 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf89172-5ff1-46b9-a7e6-2cff0d64583c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:16.262265 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf89172-5ff1-46b9-a7e6-2cff0d64583c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:16.273146 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:16.273513 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c42ef-fc17-43cd-a540-c10fddf89eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:16.274266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c42ef-fc17-43cd-a540-c10fddf89eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:16.274551 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c42ef-fc17-43cd-a540-c10fddf89eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:16.279011 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c42ef-fc17-43cd-a540-c10fddf89eb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:16.279011 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c42ef-fc17-43cd-a540-c10fddf89eb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:16.283171 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1243a-750e-41f1-9791-e6e9fa85575b admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:16.283712 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c42ef-fc17-43cd-a540-c10fddf89eb2 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:16.286113 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f0c1243a-750e-41f1-9791-e6e9fa85575b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:16.312381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1243a-750e-41f1-9791-e6e9fa85575b admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:16.323326 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed4206-07b5-4ca8-a99b-8d5944d7b8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:16.324245 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed4206-07b5-4ca8-a99b-8d5944d7b8aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:16.324528 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed4206-07b5-4ca8-a99b-8d5944d7b8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:16.325000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed4206-07b5-4ca8-a99b-8d5944d7b8aa admin admin] PATH_INFO: `/v3/users/c8c7090a5f074137b3de0794f9275b43` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:16.327438 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed4206-07b5-4ca8-a99b-8d5944d7b8aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8c7090a5f074137b3de0794f9275b43)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:16.330948 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed4206-07b5-4ca8-a99b-8d5944d7b8aa admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:16.422383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:16.460675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 980/1965] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:16 2022] POST /identity/v3/projects => generated 381 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:16.467691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c42ef-fc17-43cd-a540-c10fddf89eb2 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:16.584950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c42ef-fc17-43cd-a540-c10fddf89eb2 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:16.593372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d5d87-162b-4bbe-ae06-ee892c0990a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:16.594424 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d5d87-162b-4bbe-ae06-ee892c0990a5 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:16.594890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d5d87-162b-4bbe-ae06-ee892c0990a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:16.595439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d5d87-162b-4bbe-ae06-ee892c0990a5 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:16.596526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d5d87-162b-4bbe-ae06-ee892c0990a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:16.599304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d5d87-162b-4bbe-ae06-ee892c0990a5 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:16.612482 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-12ed4206-07b5-4ca8-a99b-8d5944d7b8aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8c7090a5f074137b3de0794f9275b43'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:16.709578 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-12ed4206-07b5-4ca8-a99b-8d5944d7b8aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8c7090a5f074137b3de0794f9275b43'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:16.710758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-12ed4206-07b5-4ca8-a99b-8d5944d7b8aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8c7090a5f074137b3de0794f9275b43'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:16.711877 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-12ed4206-07b5-4ca8-a99b-8d5944d7b8aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8c7090a5f074137b3de0794f9275b43'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:16.726820 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 981/1966] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:16 2022] POST /identity/v3/users => generated 291 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:16.729105 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d5d87-162b-4bbe-ae06-ee892c0990a5 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:16.732772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-bd1d5d87-162b-4bbe-ae06-ee892c0990a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:16.740912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-12ed4206-07b5-4ca8-a99b-8d5944d7b8aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8c7090a5f074137b3de0794f9275b43'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:16.756306 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d5d87-162b-4bbe-ae06-ee892c0990a5 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:16.766149 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6f132-2a01-442c-ae3e-2bd1bea491e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:16.767450 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6f132-2a01-442c-ae3e-2bd1bea491e9 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:16.767956 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6f132-2a01-442c-ae3e-2bd1bea491e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:16.768448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6f132-2a01-442c-ae3e-2bd1bea491e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:16.769692 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6f132-2a01-442c-ae3e-2bd1bea491e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:16.773438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6f132-2a01-442c-ae3e-2bd1bea491e9 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:16.804666 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 982/1967] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:34:16.804666 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6f132-2a01-442c-ae3e-2bd1bea491e9 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:16.808902 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 986/1968] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:16 2022] DELETE /identity/v3/users/c8c7090a5f074137b3de0794f9275b43 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:16.814796 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccaf6b4-0ffe-4a64-a3ef-b83ac672b472 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:16.815149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccaf6b4-0ffe-4a64-a3ef-b83ac672b472 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:16.815591 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccaf6b4-0ffe-4a64-a3ef-b83ac672b472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:16.830448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6f132-2a01-442c-ae3e-2bd1bea491e9 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:16.851095 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-8ccaf6b4-0ffe-4a64-a3ef-b83ac672b472 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:16.853350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8ccaf6b4-0ffe-4a64-a3ef-b83ac672b472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:16.923431 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:16.997866 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb45b02-f3c2-4b93-8ec0-8e189802c625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:16.998755 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb45b02-f3c2-4b93-8ec0-8e189802c625 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:16.999167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb45b02-f3c2-4b93-8ec0-8e189802c625 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:16.999528 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb45b02-f3c2-4b93-8ec0-8e189802c625 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:17.000556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb45b02-f3c2-4b93-8ec0-8e189802c625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:17.003708 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb45b02-f3c2-4b93-8ec0-8e189802c625 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:17.007226 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:17.024912 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 983/1969] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:17.031214 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 987/1970] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:34:17.031881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb45b02-f3c2-4b93-8ec0-8e189802c625 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:17.053035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccaf6b4-0ffe-4a64-a3ef-b83ac672b472 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:17.053903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb45b02-f3c2-4b93-8ec0-8e189802c625 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:17.058581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8ccaf6b4-0ffe-4a64-a3ef-b83ac672b472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:17.065029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbcde9-e302-4914-b473-a59cbd77c2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:17.066294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbcde9-e302-4914-b473-a59cbd77c2ba admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:17.066860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbcde9-e302-4914-b473-a59cbd77c2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:17.067468 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbcde9-e302-4914-b473-a59cbd77c2ba admin admin] PATH_INFO: `/v3/projects/f9e268b0ae4140adb428297506df2291/users/fd4bf8b9edf24f6188aef829bb009a6b/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:17.114102 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbcde9-e302-4914-b473-a59cbd77c2ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9e268b0ae4140adb428297506df2291, user_id=fd4bf8b9edf24f6188aef829bb009a6b, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:17.120440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbcde9-e302-4914-b473-a59cbd77c2ba admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:17.232731 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccaf6b4-0ffe-4a64-a3ef-b83ac672b472 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:17.254147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b977d-cf42-4bd1-a674-4bb3884d3f00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:17.259928 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b977d-cf42-4bd1-a674-4bb3884d3f00 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:17.261092 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b977d-cf42-4bd1-a674-4bb3884d3f00 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:17.265579 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b977d-cf42-4bd1-a674-4bb3884d3f00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:17.319032 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b977d-cf42-4bd1-a674-4bb3884d3f00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:17.319032 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b977d-cf42-4bd1-a674-4bb3884d3f00 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:17.339210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 988/1971] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:17.368955 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b977d-cf42-4bd1-a674-4bb3884d3f00 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:17.380691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 984/1972] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:17 2022] PUT /identity/v3/projects/f9e268b0ae4140adb428297506df2291/users/fd4bf8b9edf24f6188aef829bb009a6b/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:17.396936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-213b977d-cf42-4bd1-a674-4bb3884d3f00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:17.400446 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60599e-51af-4077-afc5-19a888900cde None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:17.401400 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60599e-51af-4077-afc5-19a888900cde None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:17.401785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60599e-51af-4077-afc5-19a888900cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:17.456085 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-3d60599e-51af-4077-afc5-19a888900cde None None] MFA Rules not processed for user `fd4bf8b9edf24f6188aef829bb009a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:17.457403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-3d60599e-51af-4077-afc5-19a888900cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:17.617372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 985/1973] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:34:17.625240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60599e-51af-4077-afc5-19a888900cde None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:17.633096 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b977d-cf42-4bd1-a674-4bb3884d3f00 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:17.745509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60599e-51af-4077-afc5-19a888900cde None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:17.759375 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fbc549-0b52-47de-8215-36eab6247e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:17.760147 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbc549-0b52-47de-8215-36eab6247e5e admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:17.760565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbc549-0b52-47de-8215-36eab6247e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:17.761777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbc549-0b52-47de-8215-36eab6247e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:17.762241 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf999f-624f-460e-8e1e-c9f840f21151 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:17.762408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fbc549-0b52-47de-8215-36eab6247e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:17.762626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf999f-624f-460e-8e1e-c9f840f21151 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:17.763095 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf999f-624f-460e-8e1e-c9f840f21151 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:17.763665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf999f-624f-460e-8e1e-c9f840f21151 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:17.765289 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fbc549-0b52-47de-8215-36eab6247e5e admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:17.767342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-78cf999f-624f-460e-8e1e-c9f840f21151 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:17.847273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf999f-624f-460e-8e1e-c9f840f21151 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:17.849354 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf999f-624f-460e-8e1e-c9f840f21151 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:17.861969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 989/1974] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:34:17.907490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 986/1975] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:17 2022] POST /identity/v3/projects => generated 383 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:17.914935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf999f-624f-460e-8e1e-c9f840f21151 service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:17.953324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf999f-624f-460e-8e1e-c9f840f21151 service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:17.955533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:18.043538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:18.141895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975d7bd-6e14-4bfd-8193-6e24545d3196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:18.144642 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975d7bd-6e14-4bfd-8193-6e24545d3196 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:18.144642 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975d7bd-6e14-4bfd-8193-6e24545d3196 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:18.145133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975d7bd-6e14-4bfd-8193-6e24545d3196 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:18.146412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6975d7bd-6e14-4bfd-8193-6e24545d3196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:18.152690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6975d7bd-6e14-4bfd-8193-6e24545d3196 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:18.322227 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 990/1976] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:17 2022] POST /identity/v3/users => generated 292 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:18.330241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fbc549-0b52-47de-8215-36eab6247e5e admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:18.359334 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975d7bd-6e14-4bfd-8193-6e24545d3196 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:18.361379 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fbc549-0b52-47de-8215-36eab6247e5e admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:18.362707 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-6975d7bd-6e14-4bfd-8193-6e24545d3196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:18.372727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd98220-d4eb-4697-9660-676a4655782d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:18.375484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd98220-d4eb-4697-9660-676a4655782d admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:18.386593 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd98220-d4eb-4697-9660-676a4655782d admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:18.387344 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd98220-d4eb-4697-9660-676a4655782d admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:18.394829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975d7bd-6e14-4bfd-8193-6e24545d3196 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:18.401577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd98220-d4eb-4697-9660-676a4655782d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:18.401860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe88d8a-7ba6-4ed5-b756-2d5308bf9d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:18.402539 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe88d8a-7ba6-4ed5-b756-2d5308bf9d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:18.402894 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe88d8a-7ba6-4ed5-b756-2d5308bf9d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:18.403283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe88d8a-7ba6-4ed5-b756-2d5308bf9d7f admin admin] PATH_INFO: `/v3/projects/2aab174a71254e998937b1fe64bea701` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:18.404088 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd98220-d4eb-4697-9660-676a4655782d admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:18.405302 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe88d8a-7ba6-4ed5-b756-2d5308bf9d7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aab174a71254e998937b1fe64bea701)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:18.407658 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe88d8a-7ba6-4ed5-b756-2d5308bf9d7f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:18.421061 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:18.445443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 987/1977] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:34:18.451664 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd98220-d4eb-4697-9660-676a4655782d admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:18.486679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd98220-d4eb-4697-9660-676a4655782d admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:18.496448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023462d-3ad8-4ea4-a4d8-6d619a9cd916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:18.497173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023462d-3ad8-4ea4-a4d8-6d619a9cd916 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:18.500739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023462d-3ad8-4ea4-a4d8-6d619a9cd916 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:18.500739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023462d-3ad8-4ea4-a4d8-6d619a9cd916 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:18.500739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023462d-3ad8-4ea4-a4d8-6d619a9cd916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:18.500739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023462d-3ad8-4ea4-a4d8-6d619a9cd916 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:18.513204 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 988/1978] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:34:18.518477 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023462d-3ad8-4ea4-a4d8-6d619a9cd916 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:18.540085 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023462d-3ad8-4ea4-a4d8-6d619a9cd916 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:18.549386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ccc73-e1e4-4cd0-b822-50e94e75fd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:18.550517 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ccc73-e1e4-4cd0-b822-50e94e75fd27 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:18.551065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ccc73-e1e4-4cd0-b822-50e94e75fd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:18.551583 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ccc73-e1e4-4cd0-b822-50e94e75fd27 admin admin] PATH_INFO: `/v3/projects/b0bdaeee125044e6a0cf1d63505949f2/users/4b77d334e33d4155857659bc27a67d52/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:18.577419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ccc73-e1e4-4cd0-b822-50e94e75fd27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0bdaeee125044e6a0cf1d63505949f2, user_id=4b77d334e33d4155857659bc27a67d52, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:18.594383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ccc73-e1e4-4cd0-b822-50e94e75fd27 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:18.619772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5fe88d8a-7ba6-4ed5-b756-2d5308bf9d7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aab174a71254e998937b1fe64bea701', 'request_id': 'req-5fe88d8a-7ba6-4ed5-b756-2d5308bf9d7f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:18.635630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5fe88d8a-7ba6-4ed5-b756-2d5308bf9d7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aab174a71254e998937b1fe64bea701', 'request_id': 'req-5fe88d8a-7ba6-4ed5-b756-2d5308bf9d7f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:18.708375 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5fe88d8a-7ba6-4ed5-b756-2d5308bf9d7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aab174a71254e998937b1fe64bea701', 'request_id': 'req-5fe88d8a-7ba6-4ed5-b756-2d5308bf9d7f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:18.730904 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 989/1979] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:18 2022] PUT /identity/v3/projects/b0bdaeee125044e6a0cf1d63505949f2/users/4b77d334e33d4155857659bc27a67d52/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:18.749091 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5fe88d8a-7ba6-4ed5-b756-2d5308bf9d7f admin admin] The token cache is being invalidate because project 2aab174a71254e998937b1fe64bea701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:18.749402 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7c10d-747b-474d-9e7a-878fff33d212 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:18.749594 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5fe88d8a-7ba6-4ed5-b756-2d5308bf9d7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:18.749956 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7c10d-747b-474d-9e7a-878fff33d212 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:18.750652 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7c10d-747b-474d-9e7a-878fff33d212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:18.752676 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5fe88d8a-7ba6-4ed5-b756-2d5308bf9d7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:18.755487 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 991/1980] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:18 2022] DELETE /identity/v3/projects/2aab174a71254e998937b1fe64bea701 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:18.763035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2472b3d-8450-4818-aa85-5d68e0b1bdbc None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:18.763514 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2472b3d-8450-4818-aa85-5d68e0b1bdbc None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:18.764004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2472b3d-8450-4818-aa85-5d68e0b1bdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:18.818393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:18.856335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-a2472b3d-8450-4818-aa85-5d68e0b1bdbc None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:18.864985 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a2472b3d-8450-4818-aa85-5d68e0b1bdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:18.871659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-5ca7c10d-747b-474d-9e7a-878fff33d212 None None] MFA Rules not processed for user `4b77d334e33d4155857659bc27a67d52`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:18.874116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5ca7c10d-747b-474d-9e7a-878fff33d212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:19.061984 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 992/1981] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:19.076882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9abb5-c62a-4bc8-a663-7989d1307a77 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:19.077350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9abb5-c62a-4bc8-a663-7989d1307a77 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:19.077824 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9abb5-c62a-4bc8-a663-7989d1307a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:19.089580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:19.094056 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 990/1982] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:19.101536 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7c10d-747b-474d-9e7a-878fff33d212 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:19.104512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5ca7c10d-747b-474d-9e7a-878fff33d212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:19.126619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-40a9abb5-c62a-4bc8-a663-7989d1307a77 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:19.128744 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-40a9abb5-c62a-4bc8-a663-7989d1307a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:19.133196 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7c10d-747b-474d-9e7a-878fff33d212 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:19.144591 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 993/1983] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:34:19.146452 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a7a05-fdfe-4ca6-97bc-88312e6cff58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:19.147905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a7a05-fdfe-4ca6-97bc-88312e6cff58 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:19.149939 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a7a05-fdfe-4ca6-97bc-88312e6cff58 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:19.150929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a7a05-fdfe-4ca6-97bc-88312e6cff58 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:19.153050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9abb5-c62a-4bc8-a663-7989d1307a77 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:19.154301 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a7a05-fdfe-4ca6-97bc-88312e6cff58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:19.159674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a7a05-fdfe-4ca6-97bc-88312e6cff58 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:19.169909 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9abb5-c62a-4bc8-a663-7989d1307a77 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:19.177169 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98692538-3b14-4e56-9294-a5879c7c33eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:19.178489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98692538-3b14-4e56-9294-a5879c7c33eb admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:19.178750 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98692538-3b14-4e56-9294-a5879c7c33eb admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:19.179876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98692538-3b14-4e56-9294-a5879c7c33eb admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:19.180467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98692538-3b14-4e56-9294-a5879c7c33eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:34:19.180995 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98692538-3b14-4e56-9294-a5879c7c33eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:19.184522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98692538-3b14-4e56-9294-a5879c7c33eb admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:19.233255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 994/1984] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:34:19.240444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98692538-3b14-4e56-9294-a5879c7c33eb admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:19.263832 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98692538-3b14-4e56-9294-a5879c7c33eb admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:19.277365 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153eada-738a-426b-8b1e-61fb9285aa11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:19.278849 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153eada-738a-426b-8b1e-61fb9285aa11 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:19.280418 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153eada-738a-426b-8b1e-61fb9285aa11 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:19.282079 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153eada-738a-426b-8b1e-61fb9285aa11 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:19.283695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153eada-738a-426b-8b1e-61fb9285aa11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:19.288474 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153eada-738a-426b-8b1e-61fb9285aa11 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:19.333337 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 991/1985] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:19 2022] POST /identity/v3/projects => generated 385 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:19.340735 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a7a05-fdfe-4ca6-97bc-88312e6cff58 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:19.364050 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:19.463270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:19.505456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a7a05-fdfe-4ca6-97bc-88312e6cff58 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:19.512123 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 995/1986] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:19 2022] POST /identity/v3/projects => generated 375 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:19.518724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153eada-738a-426b-8b1e-61fb9285aa11 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:19.645806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffec0a-2579-44a3-849f-11288e2f4471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:19.646635 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffec0a-2579-44a3-849f-11288e2f4471 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:19.647934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffec0a-2579-44a3-849f-11288e2f4471 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:19.647934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffec0a-2579-44a3-849f-11288e2f4471 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:19.648736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffec0a-2579-44a3-849f-11288e2f4471 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:19.652980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffec0a-2579-44a3-849f-11288e2f4471 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:19.687258 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153eada-738a-426b-8b1e-61fb9285aa11 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:19.698890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a2a37-5c5f-4d54-a338-2c660d12e106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:19.700066 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a2a37-5c5f-4d54-a338-2c660d12e106 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:19.700564 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a2a37-5c5f-4d54-a338-2c660d12e106 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:19.701343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a2a37-5c5f-4d54-a338-2c660d12e106 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:19.702788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a2a37-5c5f-4d54-a338-2c660d12e106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:19.706366 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a2a37-5c5f-4d54-a338-2c660d12e106 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:19.801513 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 992/1987] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:19 2022] POST /identity/v3/users => generated 299 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:34:19.803965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffec0a-2579-44a3-849f-11288e2f4471 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:19.821202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 996/1988] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:19 2022] POST /identity/v3/users => generated 288 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:19.823203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a2a37-5c5f-4d54-a338-2c660d12e106 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:19.831702 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffec0a-2579-44a3-849f-11288e2f4471 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:19.837973 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a2a37-5c5f-4d54-a338-2c660d12e106 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:19.844389 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105f536-27d3-42f4-b4f0-33007e66e271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:19.845136 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105f536-27d3-42f4-b4f0-33007e66e271 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:19.845669 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105f536-27d3-42f4-b4f0-33007e66e271 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:19.846084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105f536-27d3-42f4-b4f0-33007e66e271 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:19.846408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28659d8-f8d4-444f-9531-804bb087235d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:19.847692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105f536-27d3-42f4-b4f0-33007e66e271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:19.848211 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28659d8-f8d4-444f-9531-804bb087235d admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:19.848963 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28659d8-f8d4-444f-9531-804bb087235d admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:19.850047 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105f536-27d3-42f4-b4f0-33007e66e271 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:19.850650 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28659d8-f8d4-444f-9531-804bb087235d admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:19.853149 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28659d8-f8d4-444f-9531-804bb087235d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:19.858793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28659d8-f8d4-444f-9531-804bb087235d admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:19.870876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 997/1989] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:19.872598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105f536-27d3-42f4-b4f0-33007e66e271 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:19.886435 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105f536-27d3-42f4-b4f0-33007e66e271 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:19.894742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 993/1990] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:19.899080 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28659d8-f8d4-444f-9531-804bb087235d admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:19.920812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28659d8-f8d4-444f-9531-804bb087235d admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:19.932558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cf2c3-5505-4392-a1e3-ccdc4706a2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:19.933008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7891d59-da97-4c9b-935c-1498df48030e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:19.933579 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cf2c3-5505-4392-a1e3-ccdc4706a2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:19.934111 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cf2c3-5505-4392-a1e3-ccdc4706a2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:19.934301 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7891d59-da97-4c9b-935c-1498df48030e admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:19.934811 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7891d59-da97-4c9b-935c-1498df48030e admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:19.935311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7891d59-da97-4c9b-935c-1498df48030e admin admin] PATH_INFO: `/v3/projects/f3bfc8e70fc142daa06f7342e451d878/users/e3cfcc0f259742c891404b17d57c5b3a/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:19.936726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cf2c3-5505-4392-a1e3-ccdc4706a2c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:19.942545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267cf2c3-5505-4392-a1e3-ccdc4706a2c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:19.945737 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267cf2c3-5505-4392-a1e3-ccdc4706a2c2 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:19.974144 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7891d59-da97-4c9b-935c-1498df48030e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3bfc8e70fc142daa06f7342e451d878, user_id=e3cfcc0f259742c891404b17d57c5b3a, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:19.975131 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 998/1991] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:19.975847 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7891d59-da97-4c9b-935c-1498df48030e admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:19.976398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cf2c3-5505-4392-a1e3-ccdc4706a2c2 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:19.994784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cf2c3-5505-4392-a1e3-ccdc4706a2c2 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:20.001455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb1a9c-aabc-46c2-9eff-b7034d466fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:20.002084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb1a9c-aabc-46c2-9eff-b7034d466fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:20.002418 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb1a9c-aabc-46c2-9eff-b7034d466fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:20.002847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb1a9c-aabc-46c2-9eff-b7034d466fc4 admin admin] PATH_INFO: `/v3/projects/fa44ba6dc6284f9aa316957e3b27a262/users/f8f5abcb1be741ef9e75cb928e2d6574/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:20.009271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:20.022438 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb1a9c-aabc-46c2-9eff-b7034d466fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa44ba6dc6284f9aa316957e3b27a262, user_id=f8f5abcb1be741ef9e75cb928e2d6574, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:20.025870 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb1a9c-aabc-46c2-9eff-b7034d466fc4 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:20.087064 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 994/1992] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:19 2022] PUT /identity/v3/projects/f3bfc8e70fc142daa06f7342e451d878/users/e3cfcc0f259742c891404b17d57c5b3a/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:20.094327 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7891d59-da97-4c9b-935c-1498df48030e admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:20.185803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 999/1993] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:19 2022] PUT /identity/v3/projects/fa44ba6dc6284f9aa316957e3b27a262/users/f8f5abcb1be741ef9e75cb928e2d6574/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:20.187877 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3191a2-2785-4da9-8353-f2aa935e654f None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:20.188370 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3191a2-2785-4da9-8353-f2aa935e654f None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:20.188831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3191a2-2785-4da9-8353-f2aa935e654f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:20.201082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7891d59-da97-4c9b-935c-1498df48030e admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:20.251714 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-ab3191a2-2785-4da9-8353-f2aa935e654f None None] MFA Rules not processed for user `f8f5abcb1be741ef9e75cb928e2d6574`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:20.253102 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ab3191a2-2785-4da9-8353-f2aa935e654f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:20.362309 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3346fd6-9781-49a7-acd8-0a9316269351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:20.363284 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346fd6-9781-49a7-acd8-0a9316269351 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:20.363921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346fd6-9781-49a7-acd8-0a9316269351 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:20.364322 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346fd6-9781-49a7-acd8-0a9316269351 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:20.365439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3346fd6-9781-49a7-acd8-0a9316269351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:20.367211 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3346fd6-9781-49a7-acd8-0a9316269351 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:20.382631 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 995/1994] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:20.383648 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3346fd6-9781-49a7-acd8-0a9316269351 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:20.387261 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b3346fd6-9781-49a7-acd8-0a9316269351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:20.441111 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1000/1995] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:20.442936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3191a2-2785-4da9-8353-f2aa935e654f None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:20.471293 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3191a2-2785-4da9-8353-f2aa935e654f None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:20.483799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619f802-f2c0-46aa-95e9-2097ad10c352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:20.488855 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619f802-f2c0-46aa-95e9-2097ad10c352 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:20.489597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619f802-f2c0-46aa-95e9-2097ad10c352 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:20.490557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619f802-f2c0-46aa-95e9-2097ad10c352 admin admin] PATH_INFO: `/v3/projects/f3bfc8e70fc142daa06f7342e451d878/users/e3cfcc0f259742c891404b17d57c5b3a/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:20.495120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619f802-f2c0-46aa-95e9-2097ad10c352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3bfc8e70fc142daa06f7342e451d878, user_id=e3cfcc0f259742c891404b17d57c5b3a, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:20.500382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619f802-f2c0-46aa-95e9-2097ad10c352 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:20.512340 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:20.522088 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1001/1996] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:20 2022] PUT /identity/v3/projects/f3bfc8e70fc142daa06f7342e451d878/users/e3cfcc0f259742c891404b17d57c5b3a/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:20.523087 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3346fd6-9781-49a7-acd8-0a9316269351 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:20.524231 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619f802-f2c0-46aa-95e9-2097ad10c352 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:20.554004 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:20.666030 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9422a-11d0-4d47-be25-74d5d8a549c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:20.667328 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9422a-11d0-4d47-be25-74d5d8a549c7 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:20.667958 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9422a-11d0-4d47-be25-74d5d8a549c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:20.668643 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9422a-11d0-4d47-be25-74d5d8a549c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:20.672269 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-72a9422a-11d0-4d47-be25-74d5d8a549c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:20.698184 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619f802-f2c0-46aa-95e9-2097ad10c352 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:20.708687 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a954b-f53d-48ca-9979-524a6d31e53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:20.709850 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a954b-f53d-48ca-9979-524a6d31e53a admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:20.710262 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a954b-f53d-48ca-9979-524a6d31e53a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:20.710519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a954b-f53d-48ca-9979-524a6d31e53a admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:20.711290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a954b-f53d-48ca-9979-524a6d31e53a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:20.713443 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a954b-f53d-48ca-9979-524a6d31e53a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:20.794479 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:20.814888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9422a-11d0-4d47-be25-74d5d8a549c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:20.816848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9422a-11d0-4d47-be25-74d5d8a549c7 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:20.832855 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 996/1997] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:20.834852 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf365d-e02c-406a-99fb-478fba7e5c01 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:20.835358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf365d-e02c-406a-99fb-478fba7e5c01 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:20.835681 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf365d-e02c-406a-99fb-478fba7e5c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:20.861293 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1002/1998] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:20 2022] POST /identity/v3/projects => generated 373 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:20.862769 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a954b-f53d-48ca-9979-524a6d31e53a admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:20.892546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-13cf365d-e02c-406a-99fb-478fba7e5c01 None None] MFA Rules not processed for user `e3cfcc0f259742c891404b17d57c5b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:20.893863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-13cf365d-e02c-406a-99fb-478fba7e5c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:21.058954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 997/1999] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:21.076211 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf365d-e02c-406a-99fb-478fba7e5c01 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:21.078127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-13cf365d-e02c-406a-99fb-478fba7e5c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:21.087088 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a954b-f53d-48ca-9979-524a6d31e53a admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:21.097930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e2603-5d25-456c-92c0-82ae1fff2bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:21.100233 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e2603-5d25-456c-92c0-82ae1fff2bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:21.104972 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e2603-5d25-456c-92c0-82ae1fff2bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:21.105797 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e2603-5d25-456c-92c0-82ae1fff2bb4 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:21.109151 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e2603-5d25-456c-92c0-82ae1fff2bb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:21.113886 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e2603-5d25-456c-92c0-82ae1fff2bb4 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:21.184602 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf365d-e02c-406a-99fb-478fba7e5c01 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:21.190295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ad58d-6fdb-4a2a-952a-6fb6d8ee6b03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:21.190952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ad58d-6fdb-4a2a-952a-6fb6d8ee6b03 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:21.191279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ad58d-6fdb-4a2a-952a-6fb6d8ee6b03 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:21.191546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ad58d-6fdb-4a2a-952a-6fb6d8ee6b03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:21.197288 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d76ad58d-6fdb-4a2a-952a-6fb6d8ee6b03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:21.236125 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:21.322917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1003/2000] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:20 2022] POST /identity/v3/users => generated 293 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:21.328217 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e2603-5d25-456c-92c0-82ae1fff2bb4 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:21.330755 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ad58d-6fdb-4a2a-952a-6fb6d8ee6b03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:21.333995 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ad58d-6fdb-4a2a-952a-6fb6d8ee6b03 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:21.351196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e2603-5d25-456c-92c0-82ae1fff2bb4 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:21.351383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 998/2001] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:34:21.369116 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8343c-6219-4ee6-9d04-5abfd72270e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:21.371498 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8343c-6219-4ee6-9d04-5abfd72270e9 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:21.371950 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8343c-6219-4ee6-9d04-5abfd72270e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:21.372517 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8343c-6219-4ee6-9d04-5abfd72270e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:21.374653 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8343c-6219-4ee6-9d04-5abfd72270e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:21.379315 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8343c-6219-4ee6-9d04-5abfd72270e9 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:21.395463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1004/2002] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:21.402203 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ad58d-6fdb-4a2a-952a-6fb6d8ee6b03 service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:21.419965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ad58d-6fdb-4a2a-952a-6fb6d8ee6b03 service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:21.426255 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8117f4-9c2c-4919-b8a7-a9ac06ed6090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:21.427030 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8117f4-9c2c-4919-b8a7-a9ac06ed6090 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:21.427397 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8117f4-9c2c-4919-b8a7-a9ac06ed6090 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:21.427936 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8117f4-9c2c-4919-b8a7-a9ac06ed6090 admin admin] PATH_INFO: `/v3/projects/e12deac598d54b50abc79e15e0356f0f/users/ea6e7e07a9cd4d529ec982205c38e31d/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:21.455724 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8117f4-9c2c-4919-b8a7-a9ac06ed6090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e12deac598d54b50abc79e15e0356f0f, user_id=ea6e7e07a9cd4d529ec982205c38e31d, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:21.461179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8117f4-9c2c-4919-b8a7-a9ac06ed6090 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:21.616088 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 999/2003] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:21 2022] PUT /identity/v3/projects/e12deac598d54b50abc79e15e0356f0f/users/ea6e7e07a9cd4d529ec982205c38e31d/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:21.621967 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8343c-6219-4ee6-9d04-5abfd72270e9 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:21.643063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8343c-6219-4ee6-9d04-5abfd72270e9 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:21.825011 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe89a4f-e8e3-481a-8e52-a384b783fc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:21.826092 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe89a4f-e8e3-481a-8e52-a384b783fc5e admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:21.826839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe89a4f-e8e3-481a-8e52-a384b783fc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:21.828347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe89a4f-e8e3-481a-8e52-a384b783fc5e admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:21.828347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe89a4f-e8e3-481a-8e52-a384b783fc5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:21.830854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe89a4f-e8e3-481a-8e52-a384b783fc5e admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:21.854621 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1005/2004] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:34:21.862184 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8117f4-9c2c-4919-b8a7-a9ac06ed6090 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:21.905933 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8117f4-9c2c-4919-b8a7-a9ac06ed6090 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:21.917609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc05d1-a6ce-4a0f-8371-7bfb06ca11fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:21.920359 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc05d1-a6ce-4a0f-8371-7bfb06ca11fe admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:21.920972 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc05d1-a6ce-4a0f-8371-7bfb06ca11fe admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:21.921578 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc05d1-a6ce-4a0f-8371-7bfb06ca11fe admin admin] PATH_INFO: `/v3/projects/e12deac598d54b50abc79e15e0356f0f/users/ea6e7e07a9cd4d529ec982205c38e31d/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:21.925982 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc05d1-a6ce-4a0f-8371-7bfb06ca11fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e12deac598d54b50abc79e15e0356f0f, user_id=ea6e7e07a9cd4d529ec982205c38e31d, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:21.931479 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc05d1-a6ce-4a0f-8371-7bfb06ca11fe admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:21.959058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1000/2005] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:21 2022] PUT /identity/v3/projects/e12deac598d54b50abc79e15e0356f0f/users/ea6e7e07a9cd4d529ec982205c38e31d/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:21.974458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba11c3-c5aa-47b2-8bbb-6dc5adda3b5d None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:21.974741 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba11c3-c5aa-47b2-8bbb-6dc5adda3b5d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:21.975053 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba11c3-c5aa-47b2-8bbb-6dc5adda3b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:22.023881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-5eba11c3-c5aa-47b2-8bbb-6dc5adda3b5d None None] MFA Rules not processed for user `ea6e7e07a9cd4d529ec982205c38e31d`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:22.025487 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-5eba11c3-c5aa-47b2-8bbb-6dc5adda3b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:22.066530 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:22.231155 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:22.248658 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1006/2006] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:22.268635 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc05d1-a6ce-4a0f-8371-7bfb06ca11fe admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:22.403489 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc05d1-a6ce-4a0f-8371-7bfb06ca11fe admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:22.409872 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141c2e4-c65e-4135-b523-5f4fb13366e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:22.410401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141c2e4-c65e-4135-b523-5f4fb13366e1 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:22.411124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141c2e4-c65e-4135-b523-5f4fb13366e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:22.411871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141c2e4-c65e-4135-b523-5f4fb13366e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:22.624574 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e141c2e4-c65e-4135-b523-5f4fb13366e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:22.630418 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e141c2e4-c65e-4135-b523-5f4fb13366e1 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:22.656398 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1001/2007] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:23.317432 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:23.461547 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba11c3-c5aa-47b2-8bbb-6dc5adda3b5d None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:23.465218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-5eba11c3-c5aa-47b2-8bbb-6dc5adda3b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:23.608666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba11c3-c5aa-47b2-8bbb-6dc5adda3b5d None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:23.614351 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482497de-de78-468d-a5d3-d974794dd497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:23.615149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482497de-de78-468d-a5d3-d974794dd497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:23.615432 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482497de-de78-468d-a5d3-d974794dd497 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:23.615709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482497de-de78-468d-a5d3-d974794dd497 admin admin] PATH_INFO: `/v3/users/80341b19855e496b96b2948b916db613` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:23.617062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482497de-de78-468d-a5d3-d974794dd497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80341b19855e496b96b2948b916db613)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:23.619728 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482497de-de78-468d-a5d3-d974794dd497 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:23.933896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-482497de-de78-468d-a5d3-d974794dd497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80341b19855e496b96b2948b916db613'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:24.111988 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-482497de-de78-468d-a5d3-d974794dd497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80341b19855e496b96b2948b916db613'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:24.112971 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-482497de-de78-468d-a5d3-d974794dd497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80341b19855e496b96b2948b916db613'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:24.114564 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-482497de-de78-468d-a5d3-d974794dd497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80341b19855e496b96b2948b916db613'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:24.131043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-482497de-de78-468d-a5d3-d974794dd497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80341b19855e496b96b2948b916db613'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:24.158866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1007/2008] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:23 2022] DELETE /identity/v3/users/80341b19855e496b96b2948b916db613 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:24.439212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141c2e4-c65e-4135-b523-5f4fb13366e1 service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:24.451265 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141c2e4-c65e-4135-b523-5f4fb13366e1 service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:24.520607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482497de-de78-468d-a5d3-d974794dd497 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:24.522478 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-482497de-de78-468d-a5d3-d974794dd497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:24.532508 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fd5fd-06bd-4cbe-9e0a-24f898efe685 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:24.533808 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fd5fd-06bd-4cbe-9e0a-24f898efe685 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:24.534203 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fd5fd-06bd-4cbe-9e0a-24f898efe685 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:24.534533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fd5fd-06bd-4cbe-9e0a-24f898efe685 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:24.538434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d75fd5fd-06bd-4cbe-9e0a-24f898efe685 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:24.647423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482497de-de78-468d-a5d3-d974794dd497 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:24.655220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75fd5fd-06bd-4cbe-9e0a-24f898efe685 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:24.656553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75fd5fd-06bd-4cbe-9e0a-24f898efe685 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:24.659028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff9895-d0e1-4eb7-843b-018949692987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:24.659028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff9895-d0e1-4eb7-843b-018949692987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:24.659028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff9895-d0e1-4eb7-843b-018949692987 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:24.659670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff9895-d0e1-4eb7-843b-018949692987 admin admin] PATH_INFO: `/v3/projects/5db02cd9acdc4dc9b42a6b1ee5cc2f23` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:24.661126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff9895-d0e1-4eb7-843b-018949692987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5db02cd9acdc4dc9b42a6b1ee5cc2f23)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:24.663228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff9895-d0e1-4eb7-843b-018949692987 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:24.679849 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1002/2009] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:24.815784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a5ff9895-d0e1-4eb7-843b-018949692987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5db02cd9acdc4dc9b42a6b1ee5cc2f23', 'request_id': 'req-a5ff9895-d0e1-4eb7-843b-018949692987'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:24.828466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a5ff9895-d0e1-4eb7-843b-018949692987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5db02cd9acdc4dc9b42a6b1ee5cc2f23', 'request_id': 'req-a5ff9895-d0e1-4eb7-843b-018949692987'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:24.935984 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:24.969132 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a5ff9895-d0e1-4eb7-843b-018949692987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5db02cd9acdc4dc9b42a6b1ee5cc2f23', 'request_id': 'req-a5ff9895-d0e1-4eb7-843b-018949692987'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:24.991254 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a5ff9895-d0e1-4eb7-843b-018949692987 admin admin] The token cache is being invalidate because project 5db02cd9acdc4dc9b42a6b1ee5cc2f23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:24.996291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a5ff9895-d0e1-4eb7-843b-018949692987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:24.996291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a5ff9895-d0e1-4eb7-843b-018949692987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:24.996936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1008/2010] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:24 2022] DELETE /identity/v3/projects/5db02cd9acdc4dc9b42a6b1ee5cc2f23 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:25.007362 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e1255-5f13-4377-be81-120c2fc62780 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:25.007893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e1255-5f13-4377-be81-120c2fc62780 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:25.008354 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e1255-5f13-4377-be81-120c2fc62780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:25.040003 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c79e1255-5f13-4377-be81-120c2fc62780 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:25.042369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c79e1255-5f13-4377-be81-120c2fc62780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:25.169982 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1003/2011] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:25.196940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00424982-6df5-4d1d-9d96-f110cbdab8a6 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:25.197209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00424982-6df5-4d1d-9d96-f110cbdab8a6 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:25.197490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00424982-6df5-4d1d-9d96-f110cbdab8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:25.222399 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-00424982-6df5-4d1d-9d96-f110cbdab8a6 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:25.224768 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-00424982-6df5-4d1d-9d96-f110cbdab8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:25.238928 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1009/2012] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:25.244848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e1255-5f13-4377-be81-120c2fc62780 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:25.256558 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e1255-5f13-4377-be81-120c2fc62780 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:25.263180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d47c25-b2e9-463b-b34b-d0841cfe3bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:25.267663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d47c25-b2e9-463b-b34b-d0841cfe3bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:25.267663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d47c25-b2e9-463b-b34b-d0841cfe3bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:25.267663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d47c25-b2e9-463b-b34b-d0841cfe3bd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:25.267663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d47c25-b2e9-463b-b34b-d0841cfe3bd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:34:25.268152 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d47c25-b2e9-463b-b34b-d0841cfe3bd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:25.269593 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:25.273274 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d47c25-b2e9-463b-b34b-d0841cfe3bd7 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:25.298358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1004/2013] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:34:25.305941 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00424982-6df5-4d1d-9d96-f110cbdab8a6 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:25.323557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00424982-6df5-4d1d-9d96-f110cbdab8a6 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:25.333387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c36269-2f3d-477c-af24-816b4876f671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:25.333387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c36269-2f3d-477c-af24-816b4876f671 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:25.333806 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c36269-2f3d-477c-af24-816b4876f671 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:25.334018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c36269-2f3d-477c-af24-816b4876f671 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:25.335162 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c36269-2f3d-477c-af24-816b4876f671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:25.339517 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c36269-2f3d-477c-af24-816b4876f671 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:25.526427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1010/2014] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:25 2022] POST /identity/v3/projects => generated 367 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:25.531550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d47c25-b2e9-463b-b34b-d0841cfe3bd7 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:25.551680 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d47c25-b2e9-463b-b34b-d0841cfe3bd7 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:25.636245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee1520-4d2e-495a-b7e1-8b4aca2f9783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:25.636838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee1520-4d2e-495a-b7e1-8b4aca2f9783 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:25.637126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee1520-4d2e-495a-b7e1-8b4aca2f9783 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:25.637449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee1520-4d2e-495a-b7e1-8b4aca2f9783 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:25.638104 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee1520-4d2e-495a-b7e1-8b4aca2f9783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:25.640192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee1520-4d2e-495a-b7e1-8b4aca2f9783 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:25.834066 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1005/2015] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:25 2022] POST /identity/v3/users => generated 284 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:25.839910 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c36269-2f3d-477c-af24-816b4876f671 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:25.852219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c36269-2f3d-477c-af24-816b4876f671 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:25.863158 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1eeda-cda2-4543-925d-fcb0480675b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:25.864850 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1eeda-cda2-4543-925d-fcb0480675b7 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:25.865321 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1eeda-cda2-4543-925d-fcb0480675b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:25.865807 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1eeda-cda2-4543-925d-fcb0480675b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:25.867122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1eeda-cda2-4543-925d-fcb0480675b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:25.872204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1eeda-cda2-4543-925d-fcb0480675b7 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:25.895446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1011/2016] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:25.902029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee1520-4d2e-495a-b7e1-8b4aca2f9783 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:25.914599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee1520-4d2e-495a-b7e1-8b4aca2f9783 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:25.921124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af586b-fd72-4d62-aef3-a3aa30bdf9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:25.921985 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af586b-fd72-4d62-aef3-a3aa30bdf9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:25.922297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af586b-fd72-4d62-aef3-a3aa30bdf9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:25.922643 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af586b-fd72-4d62-aef3-a3aa30bdf9f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:25.923421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af586b-fd72-4d62-aef3-a3aa30bdf9f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:25.939884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af586b-fd72-4d62-aef3-a3aa30bdf9f8 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:25.941352 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1006/2017] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:25.947442 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1eeda-cda2-4543-925d-fcb0480675b7 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:25.959826 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1eeda-cda2-4543-925d-fcb0480675b7 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:25.966799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79916b29-f1cb-4345-a703-7881ced7183d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:25.967643 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79916b29-f1cb-4345-a703-7881ced7183d admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:25.967938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79916b29-f1cb-4345-a703-7881ced7183d admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:25.968216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79916b29-f1cb-4345-a703-7881ced7183d admin admin] PATH_INFO: `/v3/projects/b5f2cd6c39394a068674fffb52c540eb/users/529e5bfe8c8842fd8f4e6e3b8b1985b0/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:25.999244 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79916b29-f1cb-4345-a703-7881ced7183d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5f2cd6c39394a068674fffb52c540eb, user_id=529e5bfe8c8842fd8f4e6e3b8b1985b0, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:26.004716 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79916b29-f1cb-4345-a703-7881ced7183d admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:26.159185 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1012/2018] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:25 2022] PUT /identity/v3/projects/b5f2cd6c39394a068674fffb52c540eb/users/529e5bfe8c8842fd8f4e6e3b8b1985b0/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:26.168130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6c3be-2c27-40e7-933a-102507114efd None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:26.168702 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6c3be-2c27-40e7-933a-102507114efd None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:26.169487 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6c3be-2c27-40e7-933a-102507114efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:26.245088 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-47a6c3be-2c27-40e7-933a-102507114efd None None] MFA Rules not processed for user `529e5bfe8c8842fd8f4e6e3b8b1985b0`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:26.246473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-47a6c3be-2c27-40e7-933a-102507114efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:26.398603 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1007/2019] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:26.531277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:26.732055 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:26.959290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79916b29-f1cb-4345-a703-7881ced7183d admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:26.961357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-79916b29-f1cb-4345-a703-7881ced7183d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:27.039829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79916b29-f1cb-4345-a703-7881ced7183d admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:27.045452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0357c-4dac-4066-8a2a-be926c2b11c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:27.046319 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0357c-4dac-4066-8a2a-be926c2b11c4 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:27.046644 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0357c-4dac-4066-8a2a-be926c2b11c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:27.050666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0357c-4dac-4066-8a2a-be926c2b11c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:27.059891 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa0357c-4dac-4066-8a2a-be926c2b11c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:27.061732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa0357c-4dac-4066-8a2a-be926c2b11c4 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:27.075397 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1013/2020] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:34:27.269847 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:27.600249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:28.092513 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6c3be-2c27-40e7-933a-102507114efd None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:28.110641 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6c3be-2c27-40e7-933a-102507114efd None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:28.117693 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc3f41-bd9b-4415-a5e5-d6e7a41f722f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:28.120202 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc3f41-bd9b-4415-a5e5-d6e7a41f722f service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:28.120710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc3f41-bd9b-4415-a5e5-d6e7a41f722f service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:28.121205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc3f41-bd9b-4415-a5e5-d6e7a41f722f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:28.125770 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-54bc3f41-bd9b-4415-a5e5-d6e7a41f722f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:28.252785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc3f41-bd9b-4415-a5e5-d6e7a41f722f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:28.254825 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc3f41-bd9b-4415-a5e5-d6e7a41f722f service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:28.271655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1008/2021] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:34:29.032391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:30.102911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0357c-4dac-4066-8a2a-be926c2b11c4 service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:30.102911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-3fa0357c-4dac-4066-8a2a-be926c2b11c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:30.234350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:30.240964 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0357c-4dac-4066-8a2a-be926c2b11c4 service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:30.249516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba616820-e81a-4154-88cd-b58c9b01908c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:30.251513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba616820-e81a-4154-88cd-b58c9b01908c admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:30.251513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba616820-e81a-4154-88cd-b58c9b01908c admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:30.251827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba616820-e81a-4154-88cd-b58c9b01908c admin admin] PATH_INFO: `/v3/users/f8f5abcb1be741ef9e75cb928e2d6574` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:30.253878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba616820-e81a-4154-88cd-b58c9b01908c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8f5abcb1be741ef9e75cb928e2d6574)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:30.258568 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba616820-e81a-4154-88cd-b58c9b01908c admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:30.529904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-ba616820-e81a-4154-88cd-b58c9b01908c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8f5abcb1be741ef9e75cb928e2d6574'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:30.643481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-ba616820-e81a-4154-88cd-b58c9b01908c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8f5abcb1be741ef9e75cb928e2d6574'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:30.644361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-ba616820-e81a-4154-88cd-b58c9b01908c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8f5abcb1be741ef9e75cb928e2d6574'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:30.645134 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-ba616820-e81a-4154-88cd-b58c9b01908c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8f5abcb1be741ef9e75cb928e2d6574'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:30.656968 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-ba616820-e81a-4154-88cd-b58c9b01908c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8f5abcb1be741ef9e75cb928e2d6574'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:30.724173 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1014/2022] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:30 2022] DELETE /identity/v3/users/f8f5abcb1be741ef9e75cb928e2d6574 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:30.724512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449804e9-beeb-4ca8-9742-a4ca4e003821 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:30.724512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449804e9-beeb-4ca8-9742-a4ca4e003821 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:30.724512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449804e9-beeb-4ca8-9742-a4ca4e003821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:30.724512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-449804e9-beeb-4ca8-9742-a4ca4e003821 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:30.724512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-449804e9-beeb-4ca8-9742-a4ca4e003821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:30.731984 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:30.836490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1009/2023] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:30.844712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba616820-e81a-4154-88cd-b58c9b01908c admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:30.849951 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ba616820-e81a-4154-88cd-b58c9b01908c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:31.011107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449804e9-beeb-4ca8-9742-a4ca4e003821 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:31.013118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-449804e9-beeb-4ca8-9742-a4ca4e003821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:31.014877 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba616820-e81a-4154-88cd-b58c9b01908c admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:31.023106 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfbbb17-41f1-4ef7-9aeb-f5b65e91385d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:31.025327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbbb17-41f1-4ef7-9aeb-f5b65e91385d service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:31.026204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbbb17-41f1-4ef7-9aeb-f5b65e91385d service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:31.026966 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbbb17-41f1-4ef7-9aeb-f5b65e91385d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:31.035772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449804e9-beeb-4ca8-9742-a4ca4e003821 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:31.053669 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfdac1-86f0-46a5-8534-151e60594e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:31.054445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfdac1-86f0-46a5-8534-151e60594e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:31.054723 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfdac1-86f0-46a5-8534-151e60594e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:31.055036 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfdac1-86f0-46a5-8534-151e60594e50 admin admin] PATH_INFO: `/v3/users/0241c434146544039f23d3da384779f6` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:31.058062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcfdac1-86f0-46a5-8534-151e60594e50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0241c434146544039f23d3da384779f6)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:31.061533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcfdac1-86f0-46a5-8534-151e60594e50 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:31.066748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfbbb17-41f1-4ef7-9aeb-f5b65e91385d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:31.069179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfbbb17-41f1-4ef7-9aeb-f5b65e91385d service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:31.088536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1015/2024] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:31.403135 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3bcfdac1-86f0-46a5-8534-151e60594e50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0241c434146544039f23d3da384779f6'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:31.420180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3bcfdac1-86f0-46a5-8534-151e60594e50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0241c434146544039f23d3da384779f6'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:31.421482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3bcfdac1-86f0-46a5-8534-151e60594e50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0241c434146544039f23d3da384779f6'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:31.431839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3bcfdac1-86f0-46a5-8534-151e60594e50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0241c434146544039f23d3da384779f6'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:31.432645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3bcfdac1-86f0-46a5-8534-151e60594e50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0241c434146544039f23d3da384779f6'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:31.650546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1010/2025] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:31 2022] DELETE /identity/v3/users/0241c434146544039f23d3da384779f6 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:31.658035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f97c4b-0e89-4132-9afd-65a1f2a0a2cd None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:31.658035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f97c4b-0e89-4132-9afd-65a1f2a0a2cd None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:31.658035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f97c4b-0e89-4132-9afd-65a1f2a0a2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:31.681843 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-56f97c4b-0e89-4132-9afd-65a1f2a0a2cd None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:31.683336 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-56f97c4b-0e89-4132-9afd-65a1f2a0a2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:31.710367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfdac1-86f0-46a5-8534-151e60594e50 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:31.715004 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-3bcfdac1-86f0-46a5-8534-151e60594e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:31.800311 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1016/2026] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:31.821097 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f97c4b-0e89-4132-9afd-65a1f2a0a2cd None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:31.824792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-56f97c4b-0e89-4132-9afd-65a1f2a0a2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:31.849275 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:31.878165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfdac1-86f0-46a5-8534-151e60594e50 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:31.889525 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d34bfd-b52b-4a00-916f-210dc7fb1694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:31.891166 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d34bfd-b52b-4a00-916f-210dc7fb1694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:31.891687 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d34bfd-b52b-4a00-916f-210dc7fb1694 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:31.892212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d34bfd-b52b-4a00-916f-210dc7fb1694 admin admin] PATH_INFO: `/v3/projects/fa44ba6dc6284f9aa316957e3b27a262` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:31.894529 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d34bfd-b52b-4a00-916f-210dc7fb1694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa44ba6dc6284f9aa316957e3b27a262)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:31.898608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d34bfd-b52b-4a00-916f-210dc7fb1694 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:31.953401 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f97c4b-0e89-4132-9afd-65a1f2a0a2cd None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:31.958193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3266209-0839-4842-a875-ae1782e0f979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:31.958963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3266209-0839-4842-a875-ae1782e0f979 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:31.959379 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3266209-0839-4842-a875-ae1782e0f979 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:31.959743 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3266209-0839-4842-a875-ae1782e0f979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:31.974238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3266209-0839-4842-a875-ae1782e0f979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:31.982338 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3266209-0839-4842-a875-ae1782e0f979 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:31.999243 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1017/2027] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:34:32.135984 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-57d34bfd-b52b-4a00-916f-210dc7fb1694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa44ba6dc6284f9aa316957e3b27a262', 'request_id': 'req-57d34bfd-b52b-4a00-916f-210dc7fb1694'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:32.151791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-57d34bfd-b52b-4a00-916f-210dc7fb1694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa44ba6dc6284f9aa316957e3b27a262', 'request_id': 'req-57d34bfd-b52b-4a00-916f-210dc7fb1694'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:32.199325 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:32.264757 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-57d34bfd-b52b-4a00-916f-210dc7fb1694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa44ba6dc6284f9aa316957e3b27a262', 'request_id': 'req-57d34bfd-b52b-4a00-916f-210dc7fb1694'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:32.294293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-57d34bfd-b52b-4a00-916f-210dc7fb1694 admin admin] The token cache is being invalidate because project fa44ba6dc6284f9aa316957e3b27a262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:32.294293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-57d34bfd-b52b-4a00-916f-210dc7fb1694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:32.295637 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-57d34bfd-b52b-4a00-916f-210dc7fb1694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:32.299511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1011/2028] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:31 2022] DELETE /identity/v3/projects/fa44ba6dc6284f9aa316957e3b27a262 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:32.304800 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3266209-0839-4842-a875-ae1782e0f979 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:32.324346 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3266209-0839-4842-a875-ae1782e0f979 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:32.429854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb275afe-5b0a-4168-b9a6-534fa974c4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:32.430633 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb275afe-5b0a-4168-b9a6-534fa974c4c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:32.431679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb275afe-5b0a-4168-b9a6-534fa974c4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:32.432596 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb275afe-5b0a-4168-b9a6-534fa974c4c8 admin admin] PATH_INFO: `/v3/users/ea6e7e07a9cd4d529ec982205c38e31d` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:32.433677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb275afe-5b0a-4168-b9a6-534fa974c4c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea6e7e07a9cd4d529ec982205c38e31d)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:32.437058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb275afe-5b0a-4168-b9a6-534fa974c4c8 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:32.797276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-cb275afe-5b0a-4168-b9a6-534fa974c4c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea6e7e07a9cd4d529ec982205c38e31d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:32.934433 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-cb275afe-5b0a-4168-b9a6-534fa974c4c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea6e7e07a9cd4d529ec982205c38e31d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:32.936608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-cb275afe-5b0a-4168-b9a6-534fa974c4c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea6e7e07a9cd4d529ec982205c38e31d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:32.944536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-cb275afe-5b0a-4168-b9a6-534fa974c4c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea6e7e07a9cd4d529ec982205c38e31d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:32.952947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-cb275afe-5b0a-4168-b9a6-534fa974c4c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea6e7e07a9cd4d529ec982205c38e31d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:32.988560 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1018/2029] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:32 2022] DELETE /identity/v3/users/ea6e7e07a9cd4d529ec982205c38e31d => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:33.026855 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d34bfd-b52b-4a00-916f-210dc7fb1694 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:33.031945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-57d34bfd-b52b-4a00-916f-210dc7fb1694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:33.059882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d34bfd-b52b-4a00-916f-210dc7fb1694 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:33.115401 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:33.179077 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84405e73-1218-40f6-8d70-c584ce3ef093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:33.179777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84405e73-1218-40f6-8d70-c584ce3ef093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:33.180077 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84405e73-1218-40f6-8d70-c584ce3ef093 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:33.180321 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84405e73-1218-40f6-8d70-c584ce3ef093 admin admin] PATH_INFO: `/v3/projects/03b3c274843b4914aca54164afe46a34` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:33.181661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84405e73-1218-40f6-8d70-c584ce3ef093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03b3c274843b4914aca54164afe46a34)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:33.183798 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84405e73-1218-40f6-8d70-c584ce3ef093 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:33.425225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-84405e73-1218-40f6-8d70-c584ce3ef093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03b3c274843b4914aca54164afe46a34', 'request_id': 'req-84405e73-1218-40f6-8d70-c584ce3ef093'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:33.441382 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-84405e73-1218-40f6-8d70-c584ce3ef093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03b3c274843b4914aca54164afe46a34', 'request_id': 'req-84405e73-1218-40f6-8d70-c584ce3ef093'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:33.583842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-84405e73-1218-40f6-8d70-c584ce3ef093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03b3c274843b4914aca54164afe46a34', 'request_id': 'req-84405e73-1218-40f6-8d70-c584ce3ef093'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:33.614025 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-84405e73-1218-40f6-8d70-c584ce3ef093 admin admin] The token cache is being invalidate because project 03b3c274843b4914aca54164afe46a34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:33.614565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-84405e73-1218-40f6-8d70-c584ce3ef093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:33.619860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-84405e73-1218-40f6-8d70-c584ce3ef093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:33.624444 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1012/2030] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:33 2022] DELETE /identity/v3/projects/03b3c274843b4914aca54164afe46a34 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:33.631233 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb275afe-5b0a-4168-b9a6-534fa974c4c8 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:33.635579 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-cb275afe-5b0a-4168-b9a6-534fa974c4c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:33.663983 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb275afe-5b0a-4168-b9a6-534fa974c4c8 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:33.768588 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84405e73-1218-40f6-8d70-c584ce3ef093 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:33.771010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-84405e73-1218-40f6-8d70-c584ce3ef093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:33.795530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb68e18-eb80-4ef5-8bc5-1b1db979ab67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:33.797028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb68e18-eb80-4ef5-8bc5-1b1db979ab67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:33.797743 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb68e18-eb80-4ef5-8bc5-1b1db979ab67 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:33.802275 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb68e18-eb80-4ef5-8bc5-1b1db979ab67 admin admin] PATH_INFO: `/v3/users/85d420ad40a44642b6acf4edf9791e65` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:33.802856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb68e18-eb80-4ef5-8bc5-1b1db979ab67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85d420ad40a44642b6acf4edf9791e65)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:33.807981 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb68e18-eb80-4ef5-8bc5-1b1db979ab67 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:33.898691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84405e73-1218-40f6-8d70-c584ce3ef093 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:33.903921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bad802-2a82-460b-8b17-aa2abd2b6d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:33.905485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bad802-2a82-460b-8b17-aa2abd2b6d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:33.905485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bad802-2a82-460b-8b17-aa2abd2b6d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:33.905485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bad802-2a82-460b-8b17-aa2abd2b6d85 admin admin] PATH_INFO: `/v3/projects/e12deac598d54b50abc79e15e0356f0f` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:33.906464 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bad802-2a82-460b-8b17-aa2abd2b6d85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e12deac598d54b50abc79e15e0356f0f)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:33.908655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bad802-2a82-460b-8b17-aa2abd2b6d85 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:34.119345 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-71bad802-2a82-460b-8b17-aa2abd2b6d85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e12deac598d54b50abc79e15e0356f0f', 'request_id': 'req-71bad802-2a82-460b-8b17-aa2abd2b6d85'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:34.120512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5bb68e18-eb80-4ef5-8bc5-1b1db979ab67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85d420ad40a44642b6acf4edf9791e65'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:34.129077 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-71bad802-2a82-460b-8b17-aa2abd2b6d85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e12deac598d54b50abc79e15e0356f0f', 'request_id': 'req-71bad802-2a82-460b-8b17-aa2abd2b6d85'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:34.234699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5bb68e18-eb80-4ef5-8bc5-1b1db979ab67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85d420ad40a44642b6acf4edf9791e65'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:34.235392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5bb68e18-eb80-4ef5-8bc5-1b1db979ab67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85d420ad40a44642b6acf4edf9791e65'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:34.236052 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5bb68e18-eb80-4ef5-8bc5-1b1db979ab67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85d420ad40a44642b6acf4edf9791e65'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:34.246051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5bb68e18-eb80-4ef5-8bc5-1b1db979ab67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85d420ad40a44642b6acf4edf9791e65'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:34.247508 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:34.269873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1019/2031] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:33 2022] DELETE /identity/v3/users/85d420ad40a44642b6acf4edf9791e65 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:34.343299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-71bad802-2a82-460b-8b17-aa2abd2b6d85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e12deac598d54b50abc79e15e0356f0f', 'request_id': 'req-71bad802-2a82-460b-8b17-aa2abd2b6d85'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:34.356752 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-71bad802-2a82-460b-8b17-aa2abd2b6d85 admin admin] The token cache is being invalidate because project e12deac598d54b50abc79e15e0356f0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:34.357064 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-71bad802-2a82-460b-8b17-aa2abd2b6d85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:34.357746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-71bad802-2a82-460b-8b17-aa2abd2b6d85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:34.359584 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1013/2032] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:33 2022] DELETE /identity/v3/projects/e12deac598d54b50abc79e15e0356f0f => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:34.364867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7b3c4-bfa7-47b5-89a4-4f3d4a824df0 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:34.365241 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7b3c4-bfa7-47b5-89a4-4f3d4a824df0 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:34.365595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7b3c4-bfa7-47b5-89a4-4f3d4a824df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:34.388723 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-add7b3c4-bfa7-47b5-89a4-4f3d4a824df0 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:34.392455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-add7b3c4-bfa7-47b5-89a4-4f3d4a824df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:34.470246 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1020/2033] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:34:34.500989 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ea455-81ee-461f-b5ec-67e81ba433a8 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:34.501348 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ea455-81ee-461f-b5ec-67e81ba433a8 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:34.502052 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ea455-81ee-461f-b5ec-67e81ba433a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:34.542104 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-ba6ea455-81ee-461f-b5ec-67e81ba433a8 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:34.545031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ba6ea455-81ee-461f-b5ec-67e81ba433a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:34.571699 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1014/2034] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:34.578682 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7b3c4-bfa7-47b5-89a4-4f3d4a824df0 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:34.598608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7b3c4-bfa7-47b5-89a4-4f3d4a824df0 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:34.610606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f723b-a677-461b-804a-357a29bae6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:34.611729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f723b-a677-461b-804a-357a29bae6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:34.612303 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f723b-a677-461b-804a-357a29bae6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:34.612812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f723b-a677-461b-804a-357a29bae6c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:34.614108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f723b-a677-461b-804a-357a29bae6c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:34:34.614652 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f723b-a677-461b-804a-357a29bae6c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:34.617890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f723b-a677-461b-804a-357a29bae6c2 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:34.646885 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1021/2035] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:34:34.653319 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ea455-81ee-461f-b5ec-67e81ba433a8 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:34.673446 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ea455-81ee-461f-b5ec-67e81ba433a8 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:34.683996 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1b55b-37b6-45f7-81b2-d5ecb6a6172b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:34.685120 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1b55b-37b6-45f7-81b2-d5ecb6a6172b admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:34.685604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1b55b-37b6-45f7-81b2-d5ecb6a6172b admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:34.685966 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1b55b-37b6-45f7-81b2-d5ecb6a6172b admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:34.687155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1b55b-37b6-45f7-81b2-d5ecb6a6172b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:34.691251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1b55b-37b6-45f7-81b2-d5ecb6a6172b admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:34.808869 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:34.818300 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1015/2036] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:34 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:34:34.825764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f723b-a677-461b-804a-357a29bae6c2 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:34.847924 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f723b-a677-461b-804a-357a29bae6c2 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:34.947869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1b55b-37b6-45f7-81b2-d5ecb6a6172b admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:34.951665 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-61a1b55b-37b6-45f7-81b2-d5ecb6a6172b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:34.966708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cb727-67f4-4712-be82-d3bfc87504ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:34.967858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cb727-67f4-4712-be82-d3bfc87504ab admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:34.969590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cb727-67f4-4712-be82-d3bfc87504ab admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:34.969590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cb727-67f4-4712-be82-d3bfc87504ab admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:34.971403 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cb727-67f4-4712-be82-d3bfc87504ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:34.974243 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cb727-67f4-4712-be82-d3bfc87504ab admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:35.101068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1b55b-37b6-45f7-81b2-d5ecb6a6172b admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:35.109094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:35.110293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:35.110776 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:35.111263 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4 admin admin] PATH_INFO: `/v3/projects/786597f70c234b66a329c99876ea2304` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:35.113347 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=786597f70c234b66a329c99876ea2304)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:35.115909 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:35.132555 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1022/2037] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:34 2022] POST /identity/v3/users => generated 286 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:35.140394 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cb727-67f4-4712-be82-d3bfc87504ab admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:35.158780 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cb727-67f4-4712-be82-d3bfc87504ab admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:35.168368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca54b4-21ba-4189-b5e4-27938a16188d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:35.184502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca54b4-21ba-4189-b5e4-27938a16188d admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:35.184502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca54b4-21ba-4189-b5e4-27938a16188d admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:35.184502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca54b4-21ba-4189-b5e4-27938a16188d admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:35.184502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca54b4-21ba-4189-b5e4-27938a16188d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:35.184502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca54b4-21ba-4189-b5e4-27938a16188d admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:35.187981 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1023/2038] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:35.196843 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca54b4-21ba-4189-b5e4-27938a16188d admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:35.205177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca54b4-21ba-4189-b5e4-27938a16188d admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:35.211023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef74411-031c-41d6-9ac6-cc2b257402e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:35.212256 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef74411-031c-41d6-9ac6-cc2b257402e6 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:35.212603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef74411-031c-41d6-9ac6-cc2b257402e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:35.213000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef74411-031c-41d6-9ac6-cc2b257402e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:35.213781 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef74411-031c-41d6-9ac6-cc2b257402e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:35.215788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef74411-031c-41d6-9ac6-cc2b257402e6 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:35.235719 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1024/2039] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:34:35.237143 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef74411-031c-41d6-9ac6-cc2b257402e6 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:35.249508 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef74411-031c-41d6-9ac6-cc2b257402e6 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:35.256945 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1288d-b63a-4b18-9bfa-227004708d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:35.258024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1288d-b63a-4b18-9bfa-227004708d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:35.258452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1288d-b63a-4b18-9bfa-227004708d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:35.260104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1288d-b63a-4b18-9bfa-227004708d48 admin admin] PATH_INFO: `/v3/projects/8bb27292819248b199c9fdf85bcb25eb/users/3e0f4513bcd94e1d9abee989281f314a/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:35.281958 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1288d-b63a-4b18-9bfa-227004708d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bb27292819248b199c9fdf85bcb25eb, user_id=3e0f4513bcd94e1d9abee989281f314a, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:35.285446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1288d-b63a-4b18-9bfa-227004708d48 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:35.392667 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1025/2040] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:35 2022] PUT /identity/v3/projects/8bb27292819248b199c9fdf85bcb25eb/users/3e0f4513bcd94e1d9abee989281f314a/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:35.405931 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e31e41-8ede-4938-8320-074d4d188831 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:35.406499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e31e41-8ede-4938-8320-074d4d188831 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:35.407195 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e31e41-8ede-4938-8320-074d4d188831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:35.409844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '786597f70c234b66a329c99876ea2304', 'request_id': 'req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:35.424186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '786597f70c234b66a329c99876ea2304', 'request_id': 'req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:35.443043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-01e31e41-8ede-4938-8320-074d4d188831 None None] MFA Rules not processed for user `3e0f4513bcd94e1d9abee989281f314a`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:35.444410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-01e31e41-8ede-4938-8320-074d4d188831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:35.549093 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '786597f70c234b66a329c99876ea2304', 'request_id': 'req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:35.556777 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1026/2041] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:34:35.569637 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4 admin admin] The token cache is being invalidate because project 786597f70c234b66a329c99876ea2304 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:35.570048 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:35.570889 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:35.573775 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1016/2042] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:34 2022] DELETE /identity/v3/projects/786597f70c234b66a329c99876ea2304 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:35.578432 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e31e41-8ede-4938-8320-074d4d188831 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:35.578802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:35.580702 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:35.580934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-01e31e41-8ede-4938-8320-074d4d188831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:35.690656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bacb72-f7c9-4b88-a1cb-e6f8dfd9ada4 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:35.694711 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e31e41-8ede-4938-8320-074d4d188831 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:35.700071 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1abc5b-d937-4e9f-8c97-f2b2fd36ecd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:35.700417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c9f34-d471-4767-a8ce-4eb6cdaa2576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:35.700709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1abc5b-d937-4e9f-8c97-f2b2fd36ecd4 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:35.700985 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1abc5b-d937-4e9f-8c97-f2b2fd36ecd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:35.701254 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1abc5b-d937-4e9f-8c97-f2b2fd36ecd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:35.701548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c9f34-d471-4767-a8ce-4eb6cdaa2576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:35.701885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c9f34-d471-4767-a8ce-4eb6cdaa2576 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:35.702376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c9f34-d471-4767-a8ce-4eb6cdaa2576 admin admin] PATH_INFO: `/v3/users/1636d7db79ae4c04890bc8b2630647d6` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:35.704071 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-3a1abc5b-d937-4e9f-8c97-f2b2fd36ecd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:35.704491 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c9f34-d471-4767-a8ce-4eb6cdaa2576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1636d7db79ae4c04890bc8b2630647d6)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:35.708240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c9f34-d471-4767-a8ce-4eb6cdaa2576 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:35.721633 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1abc5b-d937-4e9f-8c97-f2b2fd36ecd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:35.730256 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1abc5b-d937-4e9f-8c97-f2b2fd36ecd4 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:35.743912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1027/2043] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:35.876766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:36.139012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-c07c9f34-d471-4767-a8ce-4eb6cdaa2576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1636d7db79ae4c04890bc8b2630647d6'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:36.157872 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-c07c9f34-d471-4767-a8ce-4eb6cdaa2576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1636d7db79ae4c04890bc8b2630647d6'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:36.158738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-c07c9f34-d471-4767-a8ce-4eb6cdaa2576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1636d7db79ae4c04890bc8b2630647d6'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:36.170048 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-c07c9f34-d471-4767-a8ce-4eb6cdaa2576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1636d7db79ae4c04890bc8b2630647d6'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:36.171171 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-c07c9f34-d471-4767-a8ce-4eb6cdaa2576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1636d7db79ae4c04890bc8b2630647d6'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:36.282168 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:36.295443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1017/2044] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:35 2022] DELETE /identity/v3/users/1636d7db79ae4c04890bc8b2630647d6 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:36.715242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1abc5b-d937-4e9f-8c97-f2b2fd36ecd4 service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:36.718636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-3a1abc5b-d937-4e9f-8c97-f2b2fd36ecd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:36.746825 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1abc5b-d937-4e9f-8c97-f2b2fd36ecd4 service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:36.936631 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0314b-1b22-4f11-80d5-451441e7f8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:36.938059 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0314b-1b22-4f11-80d5-451441e7f8c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:36.938749 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0314b-1b22-4f11-80d5-451441e7f8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:36.939262 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0314b-1b22-4f11-80d5-451441e7f8c3 admin admin] PATH_INFO: `/v3/users/fd4bf8b9edf24f6188aef829bb009a6b` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:36.945410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0314b-1b22-4f11-80d5-451441e7f8c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd4bf8b9edf24f6188aef829bb009a6b)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:36.952158 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0314b-1b22-4f11-80d5-451441e7f8c3 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:37.135862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:37.147532 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c9f34-d471-4767-a8ce-4eb6cdaa2576 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:37.150919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c07c9f34-d471-4767-a8ce-4eb6cdaa2576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:37.173727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c9f34-d471-4767-a8ce-4eb6cdaa2576 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:37.182088 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30446ab6-4ba6-40fd-aeac-de2601606653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:37.183610 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30446ab6-4ba6-40fd-aeac-de2601606653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:37.184370 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30446ab6-4ba6-40fd-aeac-de2601606653 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:37.187839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30446ab6-4ba6-40fd-aeac-de2601606653 admin admin] PATH_INFO: `/v3/projects/985f59ae81d046a3986d7896fdd3ba1a` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:37.187839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30446ab6-4ba6-40fd-aeac-de2601606653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=985f59ae81d046a3986d7896fdd3ba1a)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:37.191624 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30446ab6-4ba6-40fd-aeac-de2601606653 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:37.278661 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c3d0314b-1b22-4f11-80d5-451441e7f8c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd4bf8b9edf24f6188aef829bb009a6b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:37.363311 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c3d0314b-1b22-4f11-80d5-451441e7f8c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd4bf8b9edf24f6188aef829bb009a6b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:37.365430 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c3d0314b-1b22-4f11-80d5-451441e7f8c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd4bf8b9edf24f6188aef829bb009a6b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:37.367169 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c3d0314b-1b22-4f11-80d5-451441e7f8c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd4bf8b9edf24f6188aef829bb009a6b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:37.369216 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-30446ab6-4ba6-40fd-aeac-de2601606653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '985f59ae81d046a3986d7896fdd3ba1a', 'request_id': 'req-30446ab6-4ba6-40fd-aeac-de2601606653'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:37.382029 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c3d0314b-1b22-4f11-80d5-451441e7f8c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd4bf8b9edf24f6188aef829bb009a6b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:37.383102 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-30446ab6-4ba6-40fd-aeac-de2601606653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '985f59ae81d046a3986d7896fdd3ba1a', 'request_id': 'req-30446ab6-4ba6-40fd-aeac-de2601606653'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:37.389799 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:37.413167 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1028/2045] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:36 2022] DELETE /identity/v3/users/fd4bf8b9edf24f6188aef829bb009a6b => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:37.420196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041a746-7c3b-4571-a7a5-c46be689ce12 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:37.420565 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041a746-7c3b-4571-a7a5-c46be689ce12 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:37.420909 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041a746-7c3b-4571-a7a5-c46be689ce12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:37.452076 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b041a746-7c3b-4571-a7a5-c46be689ce12 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:37.454439 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b041a746-7c3b-4571-a7a5-c46be689ce12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:37.486426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-30446ab6-4ba6-40fd-aeac-de2601606653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '985f59ae81d046a3986d7896fdd3ba1a', 'request_id': 'req-30446ab6-4ba6-40fd-aeac-de2601606653'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:37.509467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-30446ab6-4ba6-40fd-aeac-de2601606653 admin admin] The token cache is being invalidate because project 985f59ae81d046a3986d7896fdd3ba1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:37.510531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-30446ab6-4ba6-40fd-aeac-de2601606653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:37.512133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-30446ab6-4ba6-40fd-aeac-de2601606653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:37.515298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1018/2046] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:37 2022] DELETE /identity/v3/projects/985f59ae81d046a3986d7896fdd3ba1a => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:37.523395 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cba712-3b8a-46c7-b5c2-401740a76f12 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:37.526336 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cba712-3b8a-46c7-b5c2-401740a76f12 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:37.529860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cba712-3b8a-46c7-b5c2-401740a76f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:37.572605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1029/2047] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:34:37.584240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-40cba712-3b8a-46c7-b5c2-401740a76f12 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:37.588718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-40cba712-3b8a-46c7-b5c2-401740a76f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:37.589678 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041a746-7c3b-4571-a7a5-c46be689ce12 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:37.591490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b041a746-7c3b-4571-a7a5-c46be689ce12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:37.607191 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1019/2048] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:34:37.628310 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ab9c6-2f41-4089-b032-d38e87302544 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:37.628767 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ab9c6-2f41-4089-b032-d38e87302544 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:37.629263 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ab9c6-2f41-4089-b032-d38e87302544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:37.641688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:37.682720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-7d6ab9c6-2f41-4089-b032-d38e87302544 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:37.685101 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-7d6ab9c6-2f41-4089-b032-d38e87302544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:37.700390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041a746-7c3b-4571-a7a5-c46be689ce12 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:37.703782 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1020/2049] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:34:37.709324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744a8e4-46fb-4f08-a2ec-79fb08512db1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:37.710658 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744a8e4-46fb-4f08-a2ec-79fb08512db1 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:37.711332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744a8e4-46fb-4f08-a2ec-79fb08512db1 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:37.712489 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744a8e4-46fb-4f08-a2ec-79fb08512db1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:37.713670 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ab9c6-2f41-4089-b032-d38e87302544 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:37.733953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744a8e4-46fb-4f08-a2ec-79fb08512db1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:37.744157 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ab9c6-2f41-4089-b032-d38e87302544 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:37.744490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744a8e4-46fb-4f08-a2ec-79fb08512db1 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:37.745120 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78929c0-f137-449a-b700-2ce2e83678e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:37.746341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78929c0-f137-449a-b700-2ce2e83678e1 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:37.746887 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78929c0-f137-449a-b700-2ce2e83678e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:37.747467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78929c0-f137-449a-b700-2ce2e83678e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:37.748748 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78929c0-f137-449a-b700-2ce2e83678e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:34:37.749486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78929c0-f137-449a-b700-2ce2e83678e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:37.752600 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78929c0-f137-449a-b700-2ce2e83678e1 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:37.754587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1030/2050] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:37.783224 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1021/2051] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:34:37.790921 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744a8e4-46fb-4f08-a2ec-79fb08512db1 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:37.813675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744a8e4-46fb-4f08-a2ec-79fb08512db1 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:37.823600 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b366d7-4776-48da-a146-34a468137ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:37.824693 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b366d7-4776-48da-a146-34a468137ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:37.825162 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b366d7-4776-48da-a146-34a468137ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:37.825538 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b366d7-4776-48da-a146-34a468137ce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:37.832928 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b366d7-4776-48da-a146-34a468137ce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:37.832928 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b366d7-4776-48da-a146-34a468137ce9 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:38.044112 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1031/2052] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:37 2022] POST /identity/v3/projects => generated 373 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:38.050765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78929c0-f137-449a-b700-2ce2e83678e1 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:38.076355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78929c0-f137-449a-b700-2ce2e83678e1 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:38.199991 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60068f8b-cc75-41f3-91bf-b7542e8d7218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:38.201666 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60068f8b-cc75-41f3-91bf-b7542e8d7218 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:38.202241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60068f8b-cc75-41f3-91bf-b7542e8d7218 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:38.202754 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60068f8b-cc75-41f3-91bf-b7542e8d7218 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:38.203735 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60068f8b-cc75-41f3-91bf-b7542e8d7218 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:38.206533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60068f8b-cc75-41f3-91bf-b7542e8d7218 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:38.331686 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1022/2053] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:38 2022] POST /identity/v3/users => generated 287 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:34:38.338625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b366d7-4776-48da-a146-34a468137ce9 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:38.362381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b366d7-4776-48da-a146-34a468137ce9 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:38.369296 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a90ed2-fa12-454e-b6a7-47016797548d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:38.370473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a90ed2-fa12-454e-b6a7-47016797548d admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:38.370829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a90ed2-fa12-454e-b6a7-47016797548d admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:38.371528 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a90ed2-fa12-454e-b6a7-47016797548d admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:38.372786 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a90ed2-fa12-454e-b6a7-47016797548d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:38.375572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a90ed2-fa12-454e-b6a7-47016797548d admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:38.395745 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1032/2054] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:38.398577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60068f8b-cc75-41f3-91bf-b7542e8d7218 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:38.408097 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a90ed2-fa12-454e-b6a7-47016797548d admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:38.411164 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b2a90ed2-fa12-454e-b6a7-47016797548d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:38.420967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60068f8b-cc75-41f3-91bf-b7542e8d7218 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:38.431213 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5160a37-192f-4e05-94fa-7884b8f36296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:38.433230 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5160a37-192f-4e05-94fa-7884b8f36296 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:38.446742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5160a37-192f-4e05-94fa-7884b8f36296 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:38.446742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5160a37-192f-4e05-94fa-7884b8f36296 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:38.446742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5160a37-192f-4e05-94fa-7884b8f36296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:38.447597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:38.448114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5160a37-192f-4e05-94fa-7884b8f36296 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:38.448329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a90ed2-fa12-454e-b6a7-47016797548d admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:38.452130 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059da7a1-d6ae-48cc-8c08-2f031f84f865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:38.453293 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059da7a1-d6ae-48cc-8c08-2f031f84f865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:38.453773 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059da7a1-d6ae-48cc-8c08-2f031f84f865 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:38.454266 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059da7a1-d6ae-48cc-8c08-2f031f84f865 admin admin] PATH_INFO: `/v3/projects/f9e268b0ae4140adb428297506df2291` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:38.456532 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059da7a1-d6ae-48cc-8c08-2f031f84f865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9e268b0ae4140adb428297506df2291)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:38.458418 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1023/2055] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:38.460453 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059da7a1-d6ae-48cc-8c08-2f031f84f865 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:38.463745 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5160a37-192f-4e05-94fa-7884b8f36296 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:38.480413 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5160a37-192f-4e05-94fa-7884b8f36296 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:38.487339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5eaa1-b9b3-4efc-b31d-dc176d5c1b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:38.488559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5eaa1-b9b3-4efc-b31d-dc176d5c1b82 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:38.489016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5eaa1-b9b3-4efc-b31d-dc176d5c1b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:38.498874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5eaa1-b9b3-4efc-b31d-dc176d5c1b82 admin admin] PATH_INFO: `/v3/projects/da30824feca04ce79489eaccbb6a7001/users/426ecc795ae243ffa7c7e1746e8fc4aa/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:38.535846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5eaa1-b9b3-4efc-b31d-dc176d5c1b82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da30824feca04ce79489eaccbb6a7001, user_id=426ecc795ae243ffa7c7e1746e8fc4aa, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:38.541429 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5eaa1-b9b3-4efc-b31d-dc176d5c1b82 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:38.682861 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1024/2056] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:38 2022] PUT /identity/v3/projects/da30824feca04ce79489eaccbb6a7001/users/426ecc795ae243ffa7c7e1746e8fc4aa/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:38.689561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d93455-ffca-45c0-aec2-536621029fe9 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:38.689900 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d93455-ffca-45c0-aec2-536621029fe9 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:38.690286 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d93455-ffca-45c0-aec2-536621029fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:38.705390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-059da7a1-d6ae-48cc-8c08-2f031f84f865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9e268b0ae4140adb428297506df2291', 'request_id': 'req-059da7a1-d6ae-48cc-8c08-2f031f84f865'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:38.713373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-75d93455-ffca-45c0-aec2-536621029fe9 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:38.714629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-75d93455-ffca-45c0-aec2-536621029fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:38.719270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-059da7a1-d6ae-48cc-8c08-2f031f84f865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9e268b0ae4140adb428297506df2291', 'request_id': 'req-059da7a1-d6ae-48cc-8c08-2f031f84f865'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:38.843842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1025/2057] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:38.858927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-059da7a1-d6ae-48cc-8c08-2f031f84f865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9e268b0ae4140adb428297506df2291', 'request_id': 'req-059da7a1-d6ae-48cc-8c08-2f031f84f865'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:38.871282 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d93455-ffca-45c0-aec2-536621029fe9 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:38.873418 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-75d93455-ffca-45c0-aec2-536621029fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:38.888035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-059da7a1-d6ae-48cc-8c08-2f031f84f865 admin admin] The token cache is being invalidate because project f9e268b0ae4140adb428297506df2291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:38.888655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-059da7a1-d6ae-48cc-8c08-2f031f84f865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:38.892078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-059da7a1-d6ae-48cc-8c08-2f031f84f865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:38.899894 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1033/2058] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:38 2022] DELETE /identity/v3/projects/f9e268b0ae4140adb428297506df2291 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:38.902705 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059da7a1-d6ae-48cc-8c08-2f031f84f865 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:38.906021 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-059da7a1-d6ae-48cc-8c08-2f031f84f865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:38.920143 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:38.939518 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059da7a1-d6ae-48cc-8c08-2f031f84f865 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:38.948298 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72749a29-0393-4d8e-8d40-233cf5ef1561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:38.949349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72749a29-0393-4d8e-8d40-233cf5ef1561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:38.949801 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72749a29-0393-4d8e-8d40-233cf5ef1561 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:38.950344 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72749a29-0393-4d8e-8d40-233cf5ef1561 admin admin] PATH_INFO: `/v3/users/4b77d334e33d4155857659bc27a67d52` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:38.952526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72749a29-0393-4d8e-8d40-233cf5ef1561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b77d334e33d4155857659bc27a67d52)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:38.956545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72749a29-0393-4d8e-8d40-233cf5ef1561 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:39.023577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d93455-ffca-45c0-aec2-536621029fe9 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:39.031115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e557dc8-0d88-440c-aef2-57095c6ed484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:39.032514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e557dc8-0d88-440c-aef2-57095c6ed484 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:39.033002 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e557dc8-0d88-440c-aef2-57095c6ed484 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:39.033518 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e557dc8-0d88-440c-aef2-57095c6ed484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:39.037141 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6e557dc8-0d88-440c-aef2-57095c6ed484 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:39.086317 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e557dc8-0d88-440c-aef2-57095c6ed484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:39.090176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e557dc8-0d88-440c-aef2-57095c6ed484 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:39.110220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1026/2059] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:34:39.202959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:39.285499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-72749a29-0393-4d8e-8d40-233cf5ef1561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b77d334e33d4155857659bc27a67d52'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:39.386913 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-72749a29-0393-4d8e-8d40-233cf5ef1561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b77d334e33d4155857659bc27a67d52'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:39.390306 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-72749a29-0393-4d8e-8d40-233cf5ef1561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b77d334e33d4155857659bc27a67d52'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:39.392040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-72749a29-0393-4d8e-8d40-233cf5ef1561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b77d334e33d4155857659bc27a67d52'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:39.409189 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-72749a29-0393-4d8e-8d40-233cf5ef1561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b77d334e33d4155857659bc27a67d52'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:39.441706 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1034/2060] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:38 2022] DELETE /identity/v3/users/4b77d334e33d4155857659bc27a67d52 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:40.096929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e557dc8-0d88-440c-aef2-57095c6ed484 service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:40.111095 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e557dc8-0d88-440c-aef2-57095c6ed484 service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:40.231755 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a882b93-87e8-4c32-9171-6008d54cffe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:40.232687 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a882b93-87e8-4c32-9171-6008d54cffe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:40.233186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a882b93-87e8-4c32-9171-6008d54cffe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:40.233611 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a882b93-87e8-4c32-9171-6008d54cffe2 admin admin] PATH_INFO: `/v3/projects/b0bdaeee125044e6a0cf1d63505949f2` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:40.235236 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a882b93-87e8-4c32-9171-6008d54cffe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0bdaeee125044e6a0cf1d63505949f2)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:40.237542 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a882b93-87e8-4c32-9171-6008d54cffe2 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:40.447001 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9a882b93-87e8-4c32-9171-6008d54cffe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0bdaeee125044e6a0cf1d63505949f2', 'request_id': 'req-9a882b93-87e8-4c32-9171-6008d54cffe2'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:40.459029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9a882b93-87e8-4c32-9171-6008d54cffe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0bdaeee125044e6a0cf1d63505949f2', 'request_id': 'req-9a882b93-87e8-4c32-9171-6008d54cffe2'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:40.581005 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9a882b93-87e8-4c32-9171-6008d54cffe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0bdaeee125044e6a0cf1d63505949f2', 'request_id': 'req-9a882b93-87e8-4c32-9171-6008d54cffe2'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:40.595955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9a882b93-87e8-4c32-9171-6008d54cffe2 admin admin] The token cache is being invalidate because project b0bdaeee125044e6a0cf1d63505949f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:40.596254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9a882b93-87e8-4c32-9171-6008d54cffe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:40.597128 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9a882b93-87e8-4c32-9171-6008d54cffe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:40.598859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1027/2061] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:40 2022] DELETE /identity/v3/projects/b0bdaeee125044e6a0cf1d63505949f2 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:40.604966 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72749a29-0393-4d8e-8d40-233cf5ef1561 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:40.608086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-72749a29-0393-4d8e-8d40-233cf5ef1561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:40.630896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72749a29-0393-4d8e-8d40-233cf5ef1561 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:40.727731 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0af72-06a0-4072-93ff-63c6858faee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:40.728768 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0af72-06a0-4072-93ff-63c6858faee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:40.729117 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0af72-06a0-4072-93ff-63c6858faee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:40.729449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0af72-06a0-4072-93ff-63c6858faee4 admin admin] PATH_INFO: `/v3/users/e3cfcc0f259742c891404b17d57c5b3a` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:40.730839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0af72-06a0-4072-93ff-63c6858faee4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3cfcc0f259742c891404b17d57c5b3a)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:40.733189 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f0af72-06a0-4072-93ff-63c6858faee4 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:40.940753 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:41.030365 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-34f0af72-06a0-4072-93ff-63c6858faee4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3cfcc0f259742c891404b17d57c5b3a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:41.138925 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a882b93-87e8-4c32-9171-6008d54cffe2 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:41.140805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9a882b93-87e8-4c32-9171-6008d54cffe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:41.160063 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a882b93-87e8-4c32-9171-6008d54cffe2 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:41.161069 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-34f0af72-06a0-4072-93ff-63c6858faee4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3cfcc0f259742c891404b17d57c5b3a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:41.162019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-34f0af72-06a0-4072-93ff-63c6858faee4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3cfcc0f259742c891404b17d57c5b3a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:41.164234 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-34f0af72-06a0-4072-93ff-63c6858faee4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3cfcc0f259742c891404b17d57c5b3a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:41.169134 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e4b9e-5978-4fe2-9a51-c5184106193f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:41.169992 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e4b9e-5978-4fe2-9a51-c5184106193f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:41.170462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e4b9e-5978-4fe2-9a51-c5184106193f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:41.170887 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e4b9e-5978-4fe2-9a51-c5184106193f admin admin] PATH_INFO: `/v3/users/3e0f4513bcd94e1d9abee989281f314a` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:41.172982 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e4b9e-5978-4fe2-9a51-c5184106193f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e0f4513bcd94e1d9abee989281f314a)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:41.175889 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e4b9e-5978-4fe2-9a51-c5184106193f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:41.181074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-34f0af72-06a0-4072-93ff-63c6858faee4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3cfcc0f259742c891404b17d57c5b3a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:41.226816 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1035/2062] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:40 2022] DELETE /identity/v3/users/e3cfcc0f259742c891404b17d57c5b3a => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:41.234050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0243a7-a5df-48e7-b382-c2ff1f965132 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:41.238043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0243a7-a5df-48e7-b382-c2ff1f965132 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:41.238043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0243a7-a5df-48e7-b382-c2ff1f965132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:41.280773 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-8a0243a7-a5df-48e7-b382-c2ff1f965132 None None] MFA Rules not processed for user `426ecc795ae243ffa7c7e1746e8fc4aa`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:41.282897 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8a0243a7-a5df-48e7-b382-c2ff1f965132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:41.404353 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1036/2063] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:41.419767 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0243a7-a5df-48e7-b382-c2ff1f965132 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:41.422026 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8a0243a7-a5df-48e7-b382-c2ff1f965132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:41.440526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:41.445099 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-bc3e4b9e-5978-4fe2-9a51-c5184106193f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e0f4513bcd94e1d9abee989281f314a'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:41.475139 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-bc3e4b9e-5978-4fe2-9a51-c5184106193f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e0f4513bcd94e1d9abee989281f314a'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:41.476425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-bc3e4b9e-5978-4fe2-9a51-c5184106193f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e0f4513bcd94e1d9abee989281f314a'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:41.492498 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-bc3e4b9e-5978-4fe2-9a51-c5184106193f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e0f4513bcd94e1d9abee989281f314a'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:41.495219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-bc3e4b9e-5978-4fe2-9a51-c5184106193f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e0f4513bcd94e1d9abee989281f314a'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:41.512456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0243a7-a5df-48e7-b382-c2ff1f965132 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:41.517651 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae14d0e-83c1-4662-b6ea-68e4f1536230 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:41.527027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae14d0e-83c1-4662-b6ea-68e4f1536230 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:41.527027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae14d0e-83c1-4662-b6ea-68e4f1536230 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:41.527027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae14d0e-83c1-4662-b6ea-68e4f1536230 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:41.527027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-4ae14d0e-83c1-4662-b6ea-68e4f1536230 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:41.537793 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae14d0e-83c1-4662-b6ea-68e4f1536230 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:41.540265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae14d0e-83c1-4662-b6ea-68e4f1536230 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:41.556968 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1037/2064] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:34:41.615947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1028/2065] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:41 2022] DELETE /identity/v3/users/3e0f4513bcd94e1d9abee989281f314a => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:41.624751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8810a-a502-489e-86c7-047da393dfc2 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:41.626491 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8810a-a502-489e-86c7-047da393dfc2 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:41.627291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8810a-a502-489e-86c7-047da393dfc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:41.667754 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-95d8810a-a502-489e-86c7-047da393dfc2 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:41.669581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-95d8810a-a502-489e-86c7-047da393dfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:41.765603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1038/2066] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:41.791276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e4b9e-5978-4fe2-9a51-c5184106193f admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:41.793860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-bc3e4b9e-5978-4fe2-9a51-c5184106193f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:41.908308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8810a-a502-489e-86c7-047da393dfc2 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:41.910350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-95d8810a-a502-489e-86c7-047da393dfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:41.932789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8810a-a502-489e-86c7-047da393dfc2 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:41.939691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e4b9e-5978-4fe2-9a51-c5184106193f admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:41.942732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b72f3-62ab-48e2-8011-3f9147830042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:41.943815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b72f3-62ab-48e2-8011-3f9147830042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:41.944396 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b72f3-62ab-48e2-8011-3f9147830042 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:41.945089 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef38c30-0f9c-45b6-be35-aad24ff502a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:41.945397 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b72f3-62ab-48e2-8011-3f9147830042 admin admin] PATH_INFO: `/v3/users/529e5bfe8c8842fd8f4e6e3b8b1985b0` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:41.945790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef38c30-0f9c-45b6-be35-aad24ff502a7 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:41.946181 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef38c30-0f9c-45b6-be35-aad24ff502a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:41.946945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef38c30-0f9c-45b6-be35-aad24ff502a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:41.949440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b72f3-62ab-48e2-8011-3f9147830042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=529e5bfe8c8842fd8f4e6e3b8b1985b0)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:41.953773 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b72f3-62ab-48e2-8011-3f9147830042 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:41.960446 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef38c30-0f9c-45b6-be35-aad24ff502a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:41.963051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef38c30-0f9c-45b6-be35-aad24ff502a7 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:41.977023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1029/2067] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:42.148948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef38c30-0f9c-45b6-be35-aad24ff502a7 service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:42.152726 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9ef38c30-0f9c-45b6-be35-aad24ff502a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:42.183020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef38c30-0f9c-45b6-be35-aad24ff502a7 service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:42.191217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26becc51-5721-4acf-8da6-6511fda31664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:42.192430 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26becc51-5721-4acf-8da6-6511fda31664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:42.193176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26becc51-5721-4acf-8da6-6511fda31664 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:42.193887 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26becc51-5721-4acf-8da6-6511fda31664 admin admin] PATH_INFO: `/v3/projects/f3bfc8e70fc142daa06f7342e451d878` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:42.196962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26becc51-5721-4acf-8da6-6511fda31664 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3bfc8e70fc142daa06f7342e451d878)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:42.201565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26becc51-5721-4acf-8da6-6511fda31664 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:42.256733 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2f0b72f3-62ab-48e2-8011-3f9147830042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '529e5bfe8c8842fd8f4e6e3b8b1985b0'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:42.408146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-26becc51-5721-4acf-8da6-6511fda31664 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3bfc8e70fc142daa06f7342e451d878', 'request_id': 'req-26becc51-5721-4acf-8da6-6511fda31664'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:42.418285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-26becc51-5721-4acf-8da6-6511fda31664 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3bfc8e70fc142daa06f7342e451d878', 'request_id': 'req-26becc51-5721-4acf-8da6-6511fda31664'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:42.455001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2f0b72f3-62ab-48e2-8011-3f9147830042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '529e5bfe8c8842fd8f4e6e3b8b1985b0'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:42.456233 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2f0b72f3-62ab-48e2-8011-3f9147830042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '529e5bfe8c8842fd8f4e6e3b8b1985b0'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:42.458154 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2f0b72f3-62ab-48e2-8011-3f9147830042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '529e5bfe8c8842fd8f4e6e3b8b1985b0'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:42.474235 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2f0b72f3-62ab-48e2-8011-3f9147830042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '529e5bfe8c8842fd8f4e6e3b8b1985b0'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:42.523200 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1039/2068] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:34:41 2022] DELETE /identity/v3/users/529e5bfe8c8842fd8f4e6e3b8b1985b0 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:42.535774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65610670-8754-4a68-99aa-8871d44b9600 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:42.536089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65610670-8754-4a68-99aa-8871d44b9600 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:42.536375 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65610670-8754-4a68-99aa-8871d44b9600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:42.574323 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-65610670-8754-4a68-99aa-8871d44b9600 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:42.576113 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-65610670-8754-4a68-99aa-8871d44b9600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:42.627445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-26becc51-5721-4acf-8da6-6511fda31664 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3bfc8e70fc142daa06f7342e451d878', 'request_id': 'req-26becc51-5721-4acf-8da6-6511fda31664'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:42.675219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-26becc51-5721-4acf-8da6-6511fda31664 admin admin] The token cache is being invalidate because project f3bfc8e70fc142daa06f7342e451d878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:42.675219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-26becc51-5721-4acf-8da6-6511fda31664 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:42.676219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-26becc51-5721-4acf-8da6-6511fda31664 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:42.684140 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1030/2069] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:42 2022] DELETE /identity/v3/projects/f3bfc8e70fc142daa06f7342e451d878 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:42.693341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858cad0-d28e-45a4-a2bf-79f9b74b27ce None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:42.695799 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858cad0-d28e-45a4-a2bf-79f9b74b27ce None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:42.697683 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858cad0-d28e-45a4-a2bf-79f9b74b27ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:42.731892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1040/2070] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:42.741501 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-3858cad0-d28e-45a4-a2bf-79f9b74b27ce None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:42.743756 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-3858cad0-d28e-45a4-a2bf-79f9b74b27ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:42.746298 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65610670-8754-4a68-99aa-8871d44b9600 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:42.748435 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-65610670-8754-4a68-99aa-8871d44b9600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:42.760167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1031/2071] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:42.773798 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e91ef-65af-4758-b822-c206f598eef1 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:42.774885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e91ef-65af-4758-b822-c206f598eef1 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:42.775446 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e91ef-65af-4758-b822-c206f598eef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:42.817084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-ca4e91ef-65af-4758-b822-c206f598eef1 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:42.820092 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ca4e91ef-65af-4758-b822-c206f598eef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:42.842021 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1032/2072] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:34:42.848946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e91ef-65af-4758-b822-c206f598eef1 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:42.857262 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65610670-8754-4a68-99aa-8871d44b9600 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:42.864720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2a926-6ab2-420b-a8e8-233714d4b1a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:42.864988 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2a926-6ab2-420b-a8e8-233714d4b1a8 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:42.865494 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2a926-6ab2-420b-a8e8-233714d4b1a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:42.865991 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2a926-6ab2-420b-a8e8-233714d4b1a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:42.876533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e91ef-65af-4758-b822-c206f598eef1 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:42.882706 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2a926-6ab2-420b-a8e8-233714d4b1a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:42.885050 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621749c-614d-4e4b-8908-0820e52d4da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:42.886093 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2a926-6ab2-420b-a8e8-233714d4b1a8 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:42.886307 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621749c-614d-4e4b-8908-0820e52d4da4 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:42.886463 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621749c-614d-4e4b-8908-0820e52d4da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:42.886860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621749c-614d-4e4b-8908-0820e52d4da4 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:42.888046 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621749c-614d-4e4b-8908-0820e52d4da4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:34:42.888555 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621749c-614d-4e4b-8908-0820e52d4da4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:42.891591 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621749c-614d-4e4b-8908-0820e52d4da4 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:42.903983 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1041/2073] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:42.919864 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1033/2074] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:34:42.922152 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2a926-6ab2-420b-a8e8-233714d4b1a8 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:42.942400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2a926-6ab2-420b-a8e8-233714d4b1a8 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:42.952476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c4553-d96b-4eab-82d3-7aedb03c39ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:42.966211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c4553-d96b-4eab-82d3-7aedb03c39ea admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:42.966211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c4553-d96b-4eab-82d3-7aedb03c39ea admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:42.966211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c4553-d96b-4eab-82d3-7aedb03c39ea admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:42.966211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c4553-d96b-4eab-82d3-7aedb03c39ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:42.966211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c4553-d96b-4eab-82d3-7aedb03c39ea admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:43.047721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621749c-614d-4e4b-8908-0820e52d4da4 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:43.052705 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2621749c-614d-4e4b-8908-0820e52d4da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:43.096671 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621749c-614d-4e4b-8908-0820e52d4da4 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:43.102628 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b5186-8609-49bd-baf9-1f7b800060ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:43.105351 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b5186-8609-49bd-baf9-1f7b800060ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:43.106048 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b5186-8609-49bd-baf9-1f7b800060ce admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:43.106599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b5186-8609-49bd-baf9-1f7b800060ce admin admin] PATH_INFO: `/v3/projects/8bb27292819248b199c9fdf85bcb25eb` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:43.109086 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b5186-8609-49bd-baf9-1f7b800060ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bb27292819248b199c9fdf85bcb25eb)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:43.112151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b5186-8609-49bd-baf9-1f7b800060ce admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:43.216667 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1042/2075] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:42 2022] POST /identity/v3/projects => generated 391 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:43.225183 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c4553-d96b-4eab-82d3-7aedb03c39ea admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:43.298147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:43.331460 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c4553-d96b-4eab-82d3-7aedb03c39ea admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:43.341990 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7b4ee-8ca8-4db2-87d1-7643bd18b86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:43.343094 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7b4ee-8ca8-4db2-87d1-7643bd18b86a admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:43.343591 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7b4ee-8ca8-4db2-87d1-7643bd18b86a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:43.344107 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7b4ee-8ca8-4db2-87d1-7643bd18b86a admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:43.345014 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7b4ee-8ca8-4db2-87d1-7643bd18b86a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:43.348608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7b4ee-8ca8-4db2-87d1-7643bd18b86a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:43.445420 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-043b5186-8609-49bd-baf9-1f7b800060ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bb27292819248b199c9fdf85bcb25eb', 'request_id': 'req-043b5186-8609-49bd-baf9-1f7b800060ce'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:43.455421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-043b5186-8609-49bd-baf9-1f7b800060ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bb27292819248b199c9fdf85bcb25eb', 'request_id': 'req-043b5186-8609-49bd-baf9-1f7b800060ce'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:43.542705 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1043/2076] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:43 2022] POST /identity/v3/users => generated 296 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:34:43.547844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7b4ee-8ca8-4db2-87d1-7643bd18b86a admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:43.559007 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7b4ee-8ca8-4db2-87d1-7643bd18b86a admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:43.574249 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-043b5186-8609-49bd-baf9-1f7b800060ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bb27292819248b199c9fdf85bcb25eb', 'request_id': 'req-043b5186-8609-49bd-baf9-1f7b800060ce'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:43.618283 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-043b5186-8609-49bd-baf9-1f7b800060ce admin admin] The token cache is being invalidate because project 8bb27292819248b199c9fdf85bcb25eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:43.618756 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-043b5186-8609-49bd-baf9-1f7b800060ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:43.623625 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-043b5186-8609-49bd-baf9-1f7b800060ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:43.628146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1034/2077] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:43 2022] DELETE /identity/v3/projects/8bb27292819248b199c9fdf85bcb25eb => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:43.638625 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da04cae-5f1b-41d2-b80b-84dfcbb1c843 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:43.643869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da04cae-5f1b-41d2-b80b-84dfcbb1c843 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:43.644314 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da04cae-5f1b-41d2-b80b-84dfcbb1c843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:43.688158 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4aa51-3e14-4476-812b-4c5a0f84e34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:43.689805 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4aa51-3e14-4476-812b-4c5a0f84e34c admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:43.690633 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4aa51-3e14-4476-812b-4c5a0f84e34c admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:43.691023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4aa51-3e14-4476-812b-4c5a0f84e34c admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:43.692319 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4aa51-3e14-4476-812b-4c5a0f84e34c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:43.696548 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b4aa51-3e14-4476-812b-4c5a0f84e34c admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:43.708217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-6da04cae-5f1b-41d2-b80b-84dfcbb1c843 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:43.709754 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6da04cae-5f1b-41d2-b80b-84dfcbb1c843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:43.716961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1044/2078] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:43.720776 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1035/2079] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:43.721763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4aa51-3e14-4476-812b-4c5a0f84e34c admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:43.724486 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a9b4aa51-3e14-4476-812b-4c5a0f84e34c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:43.729234 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfca81c-3887-4acd-aa0a-06acfb068976 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:43.729520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfca81c-3887-4acd-aa0a-06acfb068976 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:43.729830 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfca81c-3887-4acd-aa0a-06acfb068976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:43.745962 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4aa51-3e14-4476-812b-4c5a0f84e34c admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:43.755235 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be65602-7415-420e-84a0-69346b0da6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:43.756205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be65602-7415-420e-84a0-69346b0da6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:43.756413 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:43.756856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be65602-7415-420e-84a0-69346b0da6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:43.757406 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be65602-7415-420e-84a0-69346b0da6e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:43.759213 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be65602-7415-420e-84a0-69346b0da6e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:43.760057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-acfca81c-3887-4acd-aa0a-06acfb068976 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:43.761981 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be65602-7415-420e-84a0-69346b0da6e9 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:43.764388 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-acfca81c-3887-4acd-aa0a-06acfb068976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:43.780046 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1036/2080] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:34:43.786615 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1045/2081] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:34:43.786905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfca81c-3887-4acd-aa0a-06acfb068976 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:43.792659 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be65602-7415-420e-84a0-69346b0da6e9 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:43.806718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfca81c-3887-4acd-aa0a-06acfb068976 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:43.813374 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be65602-7415-420e-84a0-69346b0da6e9 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:43.816431 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10a771-3b57-4e62-aa25-992fd7235ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:43.819905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10a771-3b57-4e62-aa25-992fd7235ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:43.819905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10a771-3b57-4e62-aa25-992fd7235ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:43.819905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10a771-3b57-4e62-aa25-992fd7235ea9 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:43.822484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10a771-3b57-4e62-aa25-992fd7235ea9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:34:43.822484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10a771-3b57-4e62-aa25-992fd7235ea9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:43.824882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10a771-3b57-4e62-aa25-992fd7235ea9 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:43.825609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142f568-3c48-40c0-8671-9badec459505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:43.826558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142f568-3c48-40c0-8671-9badec459505 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:43.827073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142f568-3c48-40c0-8671-9badec459505 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:43.828050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142f568-3c48-40c0-8671-9badec459505 admin admin] PATH_INFO: `/v3/projects/18b825ade95c4fa68db820f7809d5d44/users/90b74164c1074992a71604e79047dda4/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:43.852603 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1037/2082] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:34:43.859575 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10a771-3b57-4e62-aa25-992fd7235ea9 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:43.872196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142f568-3c48-40c0-8671-9badec459505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b825ade95c4fa68db820f7809d5d44, user_id=90b74164c1074992a71604e79047dda4, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:43.877595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142f568-3c48-40c0-8671-9badec459505 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:43.889294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10a771-3b57-4e62-aa25-992fd7235ea9 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:43.897717 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090add4b-d001-43c1-923b-4981b847a996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:43.898770 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090add4b-d001-43c1-923b-4981b847a996 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:43.899931 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090add4b-d001-43c1-923b-4981b847a996 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:43.900402 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090add4b-d001-43c1-923b-4981b847a996 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:43.901634 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090add4b-d001-43c1-923b-4981b847a996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:43.905736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090add4b-d001-43c1-923b-4981b847a996 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:44.022210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1046/2083] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:43 2022] PUT /identity/v3/projects/18b825ade95c4fa68db820f7809d5d44/users/90b74164c1074992a71604e79047dda4/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:44.023513 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1038/2084] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:43 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:44.030450 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090add4b-d001-43c1-923b-4981b847a996 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:44.034911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142f568-3c48-40c0-8671-9badec459505 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:44.038621 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a142f568-3c48-40c0-8671-9badec459505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:44.131794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142f568-3c48-40c0-8671-9badec459505 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:44.138187 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97272c-0e97-4a55-92ce-d5757427ae0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:44.138937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97272c-0e97-4a55-92ce-d5757427ae0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:44.139434 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97272c-0e97-4a55-92ce-d5757427ae0f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:44.139821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97272c-0e97-4a55-92ce-d5757427ae0f admin admin] PATH_INFO: `/v3/projects/b5f2cd6c39394a068674fffb52c540eb` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:44.140979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090add4b-d001-43c1-923b-4981b847a996 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:44.141428 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97272c-0e97-4a55-92ce-d5757427ae0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5f2cd6c39394a068674fffb52c540eb)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:44.144013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97272c-0e97-4a55-92ce-d5757427ae0f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:44.148309 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00dd227-e7cc-4ae4-8e6c-046bb5c76dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:44.149839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dd227-e7cc-4ae4-8e6c-046bb5c76dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:44.150332 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dd227-e7cc-4ae4-8e6c-046bb5c76dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:44.150805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dd227-e7cc-4ae4-8e6c-046bb5c76dd8 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:44.151972 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00dd227-e7cc-4ae4-8e6c-046bb5c76dd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:44.155120 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00dd227-e7cc-4ae4-8e6c-046bb5c76dd8 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:44.318083 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1039/2085] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:44 2022] POST /identity/v3/users => generated 289 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:34:44.318083 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5b8ff-8baf-4b02-b5b9-014ad50d3b1a None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:44.318534 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5b8ff-8baf-4b02-b5b9-014ad50d3b1a None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:44.319021 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5b8ff-8baf-4b02-b5b9-014ad50d3b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:44.348005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-aa97272c-0e97-4a55-92ce-d5757427ae0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5f2cd6c39394a068674fffb52c540eb', 'request_id': 'req-aa97272c-0e97-4a55-92ce-d5757427ae0f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:44.361901 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-aa97272c-0e97-4a55-92ce-d5757427ae0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5f2cd6c39394a068674fffb52c540eb', 'request_id': 'req-aa97272c-0e97-4a55-92ce-d5757427ae0f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:44.379989 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c9e5b8ff-8baf-4b02-b5b9-014ad50d3b1a None None] MFA Rules not processed for user `90b74164c1074992a71604e79047dda4`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:44.381462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c9e5b8ff-8baf-4b02-b5b9-014ad50d3b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:44.471302 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-aa97272c-0e97-4a55-92ce-d5757427ae0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5f2cd6c39394a068674fffb52c540eb', 'request_id': 'req-aa97272c-0e97-4a55-92ce-d5757427ae0f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:44.498650 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-aa97272c-0e97-4a55-92ce-d5757427ae0f admin admin] The token cache is being invalidate because project b5f2cd6c39394a068674fffb52c540eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:34:44.499222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-aa97272c-0e97-4a55-92ce-d5757427ae0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:44.500396 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-aa97272c-0e97-4a55-92ce-d5757427ae0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:34:44.503650 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1047/2086] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:34:44 2022] DELETE /identity/v3/projects/b5f2cd6c39394a068674fffb52c540eb => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:44.509897 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97272c-0e97-4a55-92ce-d5757427ae0f admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:44.513859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1040/2087] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:44.515019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-aa97272c-0e97-4a55-92ce-d5757427ae0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:44.517305 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0cba78-3dfd-4c90-a4dd-58832a247c85 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:44.517819 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0cba78-3dfd-4c90-a4dd-58832a247c85 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:44.519450 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0cba78-3dfd-4c90-a4dd-58832a247c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:44.550339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-5a0cba78-3dfd-4c90-a4dd-58832a247c85 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:44.552666 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5a0cba78-3dfd-4c90-a4dd-58832a247c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:44.643173 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97272c-0e97-4a55-92ce-d5757427ae0f admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:44.651263 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e78952-2c53-4abd-8398-8200288b1a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:44.652333 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e78952-2c53-4abd-8398-8200288b1a90 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:44.652764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e78952-2c53-4abd-8398-8200288b1a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:44.653212 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e78952-2c53-4abd-8398-8200288b1a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:44.654324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e78952-2c53-4abd-8398-8200288b1a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:44.657556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e78952-2c53-4abd-8398-8200288b1a90 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:44.659271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1041/2088] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:44.659271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0cba78-3dfd-4c90-a4dd-58832a247c85 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:44.659271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5a0cba78-3dfd-4c90-a4dd-58832a247c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:44.677479 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1048/2089] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:44.679405 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173cdcb-f0d9-4c41-964b-20fe0084b43d None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:44.680998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173cdcb-f0d9-4c41-964b-20fe0084b43d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:44.680998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173cdcb-f0d9-4c41-964b-20fe0084b43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:44.712232 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-e173cdcb-f0d9-4c41-964b-20fe0084b43d None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:44.713986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e173cdcb-f0d9-4c41-964b-20fe0084b43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:44.724843 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1049/2090] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:44.725921 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173cdcb-f0d9-4c41-964b-20fe0084b43d None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:44.738464 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173cdcb-f0d9-4c41-964b-20fe0084b43d None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:44.745508 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a68a94-45f4-4318-8230-7750221f1369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:44.746058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a68a94-45f4-4318-8230-7750221f1369 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:44.746734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a68a94-45f4-4318-8230-7750221f1369 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:44.746734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a68a94-45f4-4318-8230-7750221f1369 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:44.747918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a68a94-45f4-4318-8230-7750221f1369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:44.750356 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a68a94-45f4-4318-8230-7750221f1369 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:44.764060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1050/2091] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:44.768637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a68a94-45f4-4318-8230-7750221f1369 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:44.772012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0cba78-3dfd-4c90-a4dd-58832a247c85 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:44.784466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b254922-9558-4488-bb1e-6e9fc8b678a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:44.785552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b254922-9558-4488-bb1e-6e9fc8b678a8 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:44.794348 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b254922-9558-4488-bb1e-6e9fc8b678a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:44.794666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a68a94-45f4-4318-8230-7750221f1369 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:44.794837 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b254922-9558-4488-bb1e-6e9fc8b678a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:44.795019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2ed43-9b3c-40d1-9278-ebfe0f00399d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:44.795019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2ed43-9b3c-40d1-9278-ebfe0f00399d admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:44.795019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2ed43-9b3c-40d1-9278-ebfe0f00399d admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:44.795474 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2ed43-9b3c-40d1-9278-ebfe0f00399d admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:44.795821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2ed43-9b3c-40d1-9278-ebfe0f00399d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:34:44.796138 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2ed43-9b3c-40d1-9278-ebfe0f00399d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:44.797861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2ed43-9b3c-40d1-9278-ebfe0f00399d admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:44.798046 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2b254922-9558-4488-bb1e-6e9fc8b678a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:44.813296 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1051/2092] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:34:44.814149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2ed43-9b3c-40d1-9278-ebfe0f00399d admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:44.819172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b254922-9558-4488-bb1e-6e9fc8b678a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:44.824022 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b254922-9558-4488-bb1e-6e9fc8b678a8 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:44.835349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2ed43-9b3c-40d1-9278-ebfe0f00399d admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:44.843328 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1042/2093] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:44.845136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b254922-9558-4488-bb1e-6e9fc8b678a8 service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:44.852628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735007eb-f3e6-4585-bbed-ccea630392dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:44.854534 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735007eb-f3e6-4585-bbed-ccea630392dc admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:44.855340 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735007eb-f3e6-4585-bbed-ccea630392dc admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:44.856072 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735007eb-f3e6-4585-bbed-ccea630392dc admin admin] PATH_INFO: `/v3/projects/e0b6de13e7c64cbdb0f43bddfaebc4ba/users/6dcde357754a4f668c30c40dc838d09f/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:44.866790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b254922-9558-4488-bb1e-6e9fc8b678a8 service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:44.877718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b302d-d537-4b9c-91b8-b4dfd1f11c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:44.879149 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b302d-d537-4b9c-91b8-b4dfd1f11c69 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:44.880019 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b302d-d537-4b9c-91b8-b4dfd1f11c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:44.880561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b302d-d537-4b9c-91b8-b4dfd1f11c69 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:44.882105 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b302d-d537-4b9c-91b8-b4dfd1f11c69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:44.885859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b302d-d537-4b9c-91b8-b4dfd1f11c69 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:44.894500 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735007eb-f3e6-4585-bbed-ccea630392dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0b6de13e7c64cbdb0f43bddfaebc4ba, user_id=6dcde357754a4f668c30c40dc838d09f, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:44.900710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735007eb-f3e6-4585-bbed-ccea630392dc admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:45.053982 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1052/2094] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:44 2022] PUT /identity/v3/projects/e0b6de13e7c64cbdb0f43bddfaebc4ba/users/6dcde357754a4f668c30c40dc838d09f/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:45.061068 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0cdb1-edd8-4824-8a20-4bc9411eda5d None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:45.061726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0cdb1-edd8-4824-8a20-4bc9411eda5d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:45.062306 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0cdb1-edd8-4824-8a20-4bc9411eda5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:45.093514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1043/2095] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:44 2022] POST /identity/v3/projects => generated 357 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:45.094461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ea7e1-9c08-48db-b3b9-4bc400b39823 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:45.095053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ea7e1-9c08-48db-b3b9-4bc400b39823 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:45.095531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ea7e1-9c08-48db-b3b9-4bc400b39823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:45.098397 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b3e0cdb1-edd8-4824-8a20-4bc9411eda5d None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:45.101104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b3e0cdb1-edd8-4824-8a20-4bc9411eda5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:45.125289 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-943ea7e1-9c08-48db-b3b9-4bc400b39823 None None] MFA Rules not processed for user `426ecc795ae243ffa7c7e1746e8fc4aa`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:45.126809 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-943ea7e1-9c08-48db-b3b9-4bc400b39823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:45.207845 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1053/2096] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:45.207845 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0cdb1-edd8-4824-8a20-4bc9411eda5d None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:45.217106 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0cdb1-edd8-4824-8a20-4bc9411eda5d None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:45.222525 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cd5f3-9a55-4154-97c3-14dfec02809f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:45.223193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cd5f3-9a55-4154-97c3-14dfec02809f admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:45.223470 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cd5f3-9a55-4154-97c3-14dfec02809f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:45.226062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cd5f3-9a55-4154-97c3-14dfec02809f admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:45.226062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cd5f3-9a55-4154-97c3-14dfec02809f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:45.226324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cd5f3-9a55-4154-97c3-14dfec02809f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:45.234174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1044/2097] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:45.234174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ea7e1-9c08-48db-b3b9-4bc400b39823 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:45.240240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-943ea7e1-9c08-48db-b3b9-4bc400b39823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:45.352294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ea7e1-9c08-48db-b3b9-4bc400b39823 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:45.358505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddb849-c8e2-43d8-be8f-6d0b335949fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:45.359604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddb849-c8e2-43d8-be8f-6d0b335949fd service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:45.360084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddb849-c8e2-43d8-be8f-6d0b335949fd service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:45.360729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddb849-c8e2-43d8-be8f-6d0b335949fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:45.380585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1054/2098] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:45 2022] POST /identity/v3/users => generated 279 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:45.380585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cd5f3-9a55-4154-97c3-14dfec02809f admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:45.380877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddb849-c8e2-43d8-be8f-6d0b335949fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:45.381645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddb849-c8e2-43d8-be8f-6d0b335949fd service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:45.391837 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cd5f3-9a55-4154-97c3-14dfec02809f admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:45.404963 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1045/2099] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:45.406263 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c3ecd-ebc4-497c-a728-ebf1e9c35251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:45.407032 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c3ecd-ebc4-497c-a728-ebf1e9c35251 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:45.407326 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c3ecd-ebc4-497c-a728-ebf1e9c35251 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:45.407609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c3ecd-ebc4-497c-a728-ebf1e9c35251 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:45.408575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c3ecd-ebc4-497c-a728-ebf1e9c35251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:45.413872 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c3ecd-ebc4-497c-a728-ebf1e9c35251 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:45.443720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1055/2100] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:45.450668 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddb849-c8e2-43d8-be8f-6d0b335949fd service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:45.465192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddb849-c8e2-43d8-be8f-6d0b335949fd service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:45.472461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12d753-e65f-442e-a420-cfa029f7d1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:45.473666 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12d753-e65f-442e-a420-cfa029f7d1b7 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:45.474115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12d753-e65f-442e-a420-cfa029f7d1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:45.474559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12d753-e65f-442e-a420-cfa029f7d1b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:45.475752 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12d753-e65f-442e-a420-cfa029f7d1b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:45.478794 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12d753-e65f-442e-a420-cfa029f7d1b7 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:45.499526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1046/2101] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:45.504416 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c3ecd-ebc4-497c-a728-ebf1e9c35251 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:45.519677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c3ecd-ebc4-497c-a728-ebf1e9c35251 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:45.534604 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fd850-c680-4d9d-8abd-fbda8cf95829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:45.536200 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fd850-c680-4d9d-8abd-fbda8cf95829 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:45.539662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fd850-c680-4d9d-8abd-fbda8cf95829 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:45.541051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fd850-c680-4d9d-8abd-fbda8cf95829 admin admin] PATH_INFO: `/v3/projects/49609873841540098752bbcf10b0c6a2/users/0ae6b623bd544095a15654cb43b0569d/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:45.582689 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867fd850-c680-4d9d-8abd-fbda8cf95829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49609873841540098752bbcf10b0c6a2, user_id=0ae6b623bd544095a15654cb43b0569d, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:45.588794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867fd850-c680-4d9d-8abd-fbda8cf95829 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:45.630400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:45.765789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1056/2102] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:45 2022] PUT /identity/v3/projects/49609873841540098752bbcf10b0c6a2/users/0ae6b623bd544095a15654cb43b0569d/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:34:45.785469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb6ccc-f113-4cbf-b79b-cfb14f2829c2 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:45.786118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb6ccc-f113-4cbf-b79b-cfb14f2829c2 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:45.786352 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb6ccc-f113-4cbf-b79b-cfb14f2829c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:45.853947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-85bb6ccc-f113-4cbf-b79b-cfb14f2829c2 None None] MFA Rules not processed for user `0ae6b623bd544095a15654cb43b0569d`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:45.855896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-85bb6ccc-f113-4cbf-b79b-cfb14f2829c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:45.877647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:46.085187 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1047/2103] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:45 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:46.092138 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fd850-c680-4d9d-8abd-fbda8cf95829 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:46.242368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fd850-c680-4d9d-8abd-fbda8cf95829 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:46.251276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694ad29-b578-4dd8-a477-632a8c56bb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:46.252410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694ad29-b578-4dd8-a477-632a8c56bb62 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:46.252912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694ad29-b578-4dd8-a477-632a8c56bb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:46.253398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694ad29-b578-4dd8-a477-632a8c56bb62 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:46.254644 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694ad29-b578-4dd8-a477-632a8c56bb62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:46.264002 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694ad29-b578-4dd8-a477-632a8c56bb62 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:46.393908 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1057/2104] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:34:46 2022] POST /identity/v3/projects => generated 355 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:46.401829 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb6ccc-f113-4cbf-b79b-cfb14f2829c2 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:46.434003 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb6ccc-f113-4cbf-b79b-cfb14f2829c2 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:46.561772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259f259-3682-4cd0-880a-2b0850125631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:46.562832 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259f259-3682-4cd0-880a-2b0850125631 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:46.563420 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259f259-3682-4cd0-880a-2b0850125631 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:46.566890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259f259-3682-4cd0-880a-2b0850125631 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:46.566890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259f259-3682-4cd0-880a-2b0850125631 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:46.569702 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259f259-3682-4cd0-880a-2b0850125631 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:46.748377 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1048/2105] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:34:46 2022] POST /identity/v3/users => generated 284 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:34:46.753406 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694ad29-b578-4dd8-a477-632a8c56bb62 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:46.767807 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694ad29-b578-4dd8-a477-632a8c56bb62 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:46.774632 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bda356-bf4a-4172-84b0-ef1fa7d7df3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:46.777446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bda356-bf4a-4172-84b0-ef1fa7d7df3c admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:46.777446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bda356-bf4a-4172-84b0-ef1fa7d7df3c admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:46.777446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bda356-bf4a-4172-84b0-ef1fa7d7df3c admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:46.778013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bda356-bf4a-4172-84b0-ef1fa7d7df3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:46.780155 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bda356-bf4a-4172-84b0-ef1fa7d7df3c admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:46.797688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1058/2106] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:34:46.804480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259f259-3682-4cd0-880a-2b0850125631 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:46.827108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259f259-3682-4cd0-880a-2b0850125631 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:46.850293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ebbb4-03b4-433f-8df0-b84b3355815c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:46.855607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ebbb4-03b4-433f-8df0-b84b3355815c admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:46.856105 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ebbb4-03b4-433f-8df0-b84b3355815c admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:46.856598 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ebbb4-03b4-433f-8df0-b84b3355815c admin admin] PATH_INFO: `/v3/projects/746f576fcab94636bf74991859cefaad/users/e791862730944cb2aac31d261d1dec31/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:46.868390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:46.896994 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ebbb4-03b4-433f-8df0-b84b3355815c admin admin] RBAC: Authorizing `identity:create_grant(project_id=746f576fcab94636bf74991859cefaad, user_id=e791862730944cb2aac31d261d1dec31, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:46.905219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ebbb4-03b4-433f-8df0-b84b3355815c admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:47.056445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1049/2107] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:46 2022] PUT /identity/v3/projects/746f576fcab94636bf74991859cefaad/users/e791862730944cb2aac31d261d1dec31/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:47.061916 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bda356-bf4a-4172-84b0-ef1fa7d7df3c admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:47.088305 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bda356-bf4a-4172-84b0-ef1fa7d7df3c admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:47.220137 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993fd8ed-83b2-4fae-93e6-bbaba527014f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:47.221367 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fd8ed-83b2-4fae-93e6-bbaba527014f admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:47.221933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fd8ed-83b2-4fae-93e6-bbaba527014f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:47.222771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fd8ed-83b2-4fae-93e6-bbaba527014f admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:47.223773 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993fd8ed-83b2-4fae-93e6-bbaba527014f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:47.229745 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993fd8ed-83b2-4fae-93e6-bbaba527014f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:47.245266 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1059/2108] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:34:47.246760 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ebbb4-03b4-433f-8df0-b84b3355815c admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:47.264571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ebbb4-03b4-433f-8df0-b84b3355815c admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:47.288198 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a41e5-d143-42f8-9509-0891517460f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:47.289629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a41e5-d143-42f8-9509-0891517460f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:47.290030 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a41e5-d143-42f8-9509-0891517460f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:47.290269 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a41e5-d143-42f8-9509-0891517460f5 admin admin] PATH_INFO: `/v3/projects/746f576fcab94636bf74991859cefaad/users/e791862730944cb2aac31d261d1dec31/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:47.297316 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a41e5-d143-42f8-9509-0891517460f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=746f576fcab94636bf74991859cefaad, user_id=e791862730944cb2aac31d261d1dec31, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:47.304554 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a41e5-d143-42f8-9509-0891517460f5 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:47.327587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1050/2109] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:34:47 2022] PUT /identity/v3/projects/746f576fcab94636bf74991859cefaad/users/e791862730944cb2aac31d261d1dec31/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:34:47.359133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00635087-4577-4058-8959-b0263e7e89bb None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:47.359633 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00635087-4577-4058-8959-b0263e7e89bb None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:47.360135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00635087-4577-4058-8959-b0263e7e89bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:47.415953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-00635087-4577-4058-8959-b0263e7e89bb None None] MFA Rules not processed for user `e791862730944cb2aac31d261d1dec31`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:47.419734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-00635087-4577-4058-8959-b0263e7e89bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:47.520262 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1060/2110] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:47 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:47.684317 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:47.851078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010763bd-f52e-4d23-bf25-0ffab216b367 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:47.852347 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010763bd-f52e-4d23-bf25-0ffab216b367 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:47.852903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010763bd-f52e-4d23-bf25-0ffab216b367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:47.893645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-010763bd-f52e-4d23-bf25-0ffab216b367 None None] MFA Rules not processed for user `6dcde357754a4f668c30c40dc838d09f`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:47.895385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-010763bd-f52e-4d23-bf25-0ffab216b367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:48.028030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00635087-4577-4058-8959-b0263e7e89bb None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:48.030682 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-00635087-4577-4058-8959-b0263e7e89bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:48.075945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1051/2111] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:34:48.125201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00635087-4577-4058-8959-b0263e7e89bb None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:48.129549 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e966811-f6b2-4741-9ea4-d606a1d475c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:48.130222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e966811-f6b2-4741-9ea4-d606a1d475c7 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:48.130503 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e966811-f6b2-4741-9ea4-d606a1d475c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:48.130832 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e966811-f6b2-4741-9ea4-d606a1d475c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:48.143515 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e966811-f6b2-4741-9ea4-d606a1d475c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:48.145757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e966811-f6b2-4741-9ea4-d606a1d475c7 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:48.159648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1061/2112] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:48 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:48.359323 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:48.376849 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010763bd-f52e-4d23-bf25-0ffab216b367 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:48.390268 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010763bd-f52e-4d23-bf25-0ffab216b367 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:48.394340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b7c59-d0b4-4d41-8efb-e6608bab5b0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:48.395075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b7c59-d0b4-4d41-8efb-e6608bab5b0f service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:48.395356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b7c59-d0b4-4d41-8efb-e6608bab5b0f service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:48.397407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b7c59-d0b4-4d41-8efb-e6608bab5b0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:48.406133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b7c59-d0b4-4d41-8efb-e6608bab5b0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:48.407866 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b7c59-d0b4-4d41-8efb-e6608bab5b0f service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:48.421625 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1052/2113] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:48.630654 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e966811-f6b2-4741-9ea4-d606a1d475c7 service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:34:48.641731 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e966811-f6b2-4741-9ea4-d606a1d475c7 service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:34:48.647163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da7f05-fea9-4e82-b55a-a281eec625d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:34:48.647347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da7f05-fea9-4e82-b55a-a281eec625d0 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:48.647501 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da7f05-fea9-4e82-b55a-a281eec625d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:48.647775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da7f05-fea9-4e82-b55a-a281eec625d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:48.699799 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:48.720843 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da7f05-fea9-4e82-b55a-a281eec625d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:34:48.722392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da7f05-fea9-4e82-b55a-a281eec625d0 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:34:48.734456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1062/2114] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:34:48 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:34:49.461969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:50.648444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:34:52.185119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36fc40b-a06b-4584-9031-c21ae36e6e1e None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:34:52.185702 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36fc40b-a06b-4584-9031-c21ae36e6e1e None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:34:52.185702 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36fc40b-a06b-4584-9031-c21ae36e6e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:34:52.212807 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-e36fc40b-a06b-4584-9031-c21ae36e6e1e None None] MFA Rules not processed for user `6dcde357754a4f668c30c40dc838d09f`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:34:52.214311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e36fc40b-a06b-4584-9031-c21ae36e6e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:34:52.234929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1053/2115] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:34:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:01.092256 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=82 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:02.203917 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:11.918271 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da7f05-fea9-4e82-b55a-a281eec625d0 service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:11.920436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-06da7f05-fea9-4e82-b55a-a281eec625d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:12.089360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da7f05-fea9-4e82-b55a-a281eec625d0 service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:12.100889 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986cc1b-dcbb-44af-8c40-3f84be702d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:12.104185 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986cc1b-dcbb-44af-8c40-3f84be702d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:12.105296 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986cc1b-dcbb-44af-8c40-3f84be702d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:12.107407 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986cc1b-dcbb-44af-8c40-3f84be702d14 admin admin] PATH_INFO: `/v3/users/90b74164c1074992a71604e79047dda4` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:12.110637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986cc1b-dcbb-44af-8c40-3f84be702d14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90b74164c1074992a71604e79047dda4)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:12.117970 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986cc1b-dcbb-44af-8c40-3f84be702d14 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:12.183805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369065e-3982-49e6-ab64-f39d36187d38 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:12.184242 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369065e-3982-49e6-ab64-f39d36187d38 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:12.184361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369065e-3982-49e6-ab64-f39d36187d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:12.240426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-1369065e-3982-49e6-ab64-f39d36187d38 None None] MFA Rules not processed for user `a145e63a44db43acafc92bb2a238f463`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:12.246801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-1369065e-3982-49e6-ab64-f39d36187d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:12.402173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1054/2116] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:12.509573 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c986cc1b-dcbb-44af-8c40-3f84be702d14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90b74164c1074992a71604e79047dda4'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:12.670006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c986cc1b-dcbb-44af-8c40-3f84be702d14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90b74164c1074992a71604e79047dda4'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:12.671032 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c986cc1b-dcbb-44af-8c40-3f84be702d14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90b74164c1074992a71604e79047dda4'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:12.673110 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c986cc1b-dcbb-44af-8c40-3f84be702d14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90b74164c1074992a71604e79047dda4'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:12.701738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c986cc1b-dcbb-44af-8c40-3f84be702d14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90b74164c1074992a71604e79047dda4'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:12.758890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1063/2117] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:35:11 2022] DELETE /identity/v3/users/90b74164c1074992a71604e79047dda4 => generated 0 bytes in 841 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:12.770560 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db9778-7ff2-40ee-a626-cdd7a04b87d5 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:12.771112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db9778-7ff2-40ee-a626-cdd7a04b87d5 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:12.771575 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db9778-7ff2-40ee-a626-cdd7a04b87d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:12.818490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-b6db9778-7ff2-40ee-a626-cdd7a04b87d5 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:12.820980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b6db9778-7ff2-40ee-a626-cdd7a04b87d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:13.002531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1055/2118] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:13.022936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986cc1b-dcbb-44af-8c40-3f84be702d14 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:13.028112 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c986cc1b-dcbb-44af-8c40-3f84be702d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:13.144739 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986cc1b-dcbb-44af-8c40-3f84be702d14 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:13.156822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb2516-5fa2-4cb1-ae34-ac5952dda390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:13.157831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb2516-5fa2-4cb1-ae34-ac5952dda390 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:13.158342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb2516-5fa2-4cb1-ae34-ac5952dda390 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:13.158696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb2516-5fa2-4cb1-ae34-ac5952dda390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:13.186546 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb2516-5fa2-4cb1-ae34-ac5952dda390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:13.190094 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb2516-5fa2-4cb1-ae34-ac5952dda390 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:13.215147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1064/2119] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:35:14.355606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db9778-7ff2-40ee-a626-cdd7a04b87d5 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:14.359987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b6db9778-7ff2-40ee-a626-cdd7a04b87d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:14.395167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db9778-7ff2-40ee-a626-cdd7a04b87d5 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:14.417650 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5736660-76c3-4dc2-b63d-f28a4cf87ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:14.418905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5736660-76c3-4dc2-b63d-f28a4cf87ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:14.419661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5736660-76c3-4dc2-b63d-f28a4cf87ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:14.420754 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5736660-76c3-4dc2-b63d-f28a4cf87ccc admin admin] PATH_INFO: `/v3/projects/18b825ade95c4fa68db820f7809d5d44` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:14.424039 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5736660-76c3-4dc2-b63d-f28a4cf87ccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=18b825ade95c4fa68db820f7809d5d44)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:14.436451 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5736660-76c3-4dc2-b63d-f28a4cf87ccc admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:14.748635 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e5736660-76c3-4dc2-b63d-f28a4cf87ccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18b825ade95c4fa68db820f7809d5d44', 'request_id': 'req-e5736660-76c3-4dc2-b63d-f28a4cf87ccc'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:14.775006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e5736660-76c3-4dc2-b63d-f28a4cf87ccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18b825ade95c4fa68db820f7809d5d44', 'request_id': 'req-e5736660-76c3-4dc2-b63d-f28a4cf87ccc'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:14.925953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e5736660-76c3-4dc2-b63d-f28a4cf87ccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18b825ade95c4fa68db820f7809d5d44', 'request_id': 'req-e5736660-76c3-4dc2-b63d-f28a4cf87ccc'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:14.946598 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e5736660-76c3-4dc2-b63d-f28a4cf87ccc admin admin] The token cache is being invalidate because project 18b825ade95c4fa68db820f7809d5d44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:35:14.947012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e5736660-76c3-4dc2-b63d-f28a4cf87ccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:14.947910 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e5736660-76c3-4dc2-b63d-f28a4cf87ccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:14.951271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1056/2120] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:35:14 2022] DELETE /identity/v3/projects/18b825ade95c4fa68db820f7809d5d44 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:18.066250 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72eb61-d3b8-4b6e-8cdf-c808eae28f16 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:18.067152 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72eb61-d3b8-4b6e-8cdf-c808eae28f16 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:18.067388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72eb61-d3b8-4b6e-8cdf-c808eae28f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:18.136379 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-8e72eb61-d3b8-4b6e-8cdf-c808eae28f16 None None] MFA Rules not processed for user `c5e931aa22344990a0f032159ebc20ad`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:18.141404 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8e72eb61-d3b8-4b6e-8cdf-c808eae28f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:18.260065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:18.369367 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1065/2121] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:35:18.655203 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acbee1-354c-46f8-980c-16a7a207e0c1 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:18.656104 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acbee1-354c-46f8-980c-16a7a207e0c1 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:18.656938 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acbee1-354c-46f8-980c-16a7a207e0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:18.692708 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-98acbee1-354c-46f8-980c-16a7a207e0c1 None None] MFA Rules not processed for user `8121ba4cf2cf44d9a5a55975ea85928c`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:18.694671 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-98acbee1-354c-46f8-980c-16a7a207e0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:18.756422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:18.812514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1057/2122] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 7 13:35:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:18.872807 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72eb61-d3b8-4b6e-8cdf-c808eae28f16 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:18.879301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8e72eb61-d3b8-4b6e-8cdf-c808eae28f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:19.034547 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72eb61-d3b8-4b6e-8cdf-c808eae28f16 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:19.053685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4999994-6a99-4441-b9ee-2df706d86546 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cedc7091d2f46b5be68818d26399e9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:19.053685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4999994-6a99-4441-b9ee-2df706d86546 service swift] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:19.053685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4999994-6a99-4441-b9ee-2df706d86546 service swift] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:19.053685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4999994-6a99-4441-b9ee-2df706d86546 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:19.073833 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4999994-6a99-4441-b9ee-2df706d86546 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:19.075918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4999994-6a99-4441-b9ee-2df706d86546 service swift] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:19.093526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1066/2123] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 7 13:35:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:35:24.104809 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acbee1-354c-46f8-980c-16a7a207e0c1 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:24.107009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-98acbee1-354c-46f8-980c-16a7a207e0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:24.342017 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acbee1-354c-46f8-980c-16a7a207e0c1 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:24.348513 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e0263-6b45-4dad-9ba0-e5554ef70da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:24.349604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e0263-6b45-4dad-9ba0-e5554ef70da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:24.350461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e0263-6b45-4dad-9ba0-e5554ef70da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:24.352043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e0263-6b45-4dad-9ba0-e5554ef70da1 admin admin] PATH_INFO: `/v3/users/0ae6b623bd544095a15654cb43b0569d` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:24.359305 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e0263-6b45-4dad-9ba0-e5554ef70da1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ae6b623bd544095a15654cb43b0569d)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:24.362769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e0263-6b45-4dad-9ba0-e5554ef70da1 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:24.656953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ab8e0263-6b45-4dad-9ba0-e5554ef70da1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ae6b623bd544095a15654cb43b0569d'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:24.714086 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ab8e0263-6b45-4dad-9ba0-e5554ef70da1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ae6b623bd544095a15654cb43b0569d'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:24.714865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ab8e0263-6b45-4dad-9ba0-e5554ef70da1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ae6b623bd544095a15654cb43b0569d'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:24.737032 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ab8e0263-6b45-4dad-9ba0-e5554ef70da1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ae6b623bd544095a15654cb43b0569d'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:24.740053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ab8e0263-6b45-4dad-9ba0-e5554ef70da1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ae6b623bd544095a15654cb43b0569d'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:24.861284 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1058/2124] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:35:24 2022] DELETE /identity/v3/users/0ae6b623bd544095a15654cb43b0569d => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:24.861779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756d7d3-cd99-486f-9807-1d28bb59b24e None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:24.862988 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756d7d3-cd99-486f-9807-1d28bb59b24e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:24.863587 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756d7d3-cd99-486f-9807-1d28bb59b24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:24.944405 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-2756d7d3-cd99-486f-9807-1d28bb59b24e None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:24.946574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-2756d7d3-cd99-486f-9807-1d28bb59b24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:25.114946 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1067/2125] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:25.152162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e0263-6b45-4dad-9ba0-e5554ef70da1 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:25.159542 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ab8e0263-6b45-4dad-9ba0-e5554ef70da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:25.346958 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e0263-6b45-4dad-9ba0-e5554ef70da1 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:25.368779 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c99b8e-68c3-4aa0-a539-7b3a050ce80f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:25.368779 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c99b8e-68c3-4aa0-a539-7b3a050ce80f service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:25.368779 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c99b8e-68c3-4aa0-a539-7b3a050ce80f service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:25.368779 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c99b8e-68c3-4aa0-a539-7b3a050ce80f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:25.382097 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:25.386195 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c99b8e-68c3-4aa0-a539-7b3a050ce80f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:25.387836 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c99b8e-68c3-4aa0-a539-7b3a050ce80f service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:25.427346 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1059/2126] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:35:27.366358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756d7d3-cd99-486f-9807-1d28bb59b24e None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:27.368678 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-2756d7d3-cd99-486f-9807-1d28bb59b24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:27.394112 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756d7d3-cd99-486f-9807-1d28bb59b24e None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:27.427854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865a76f-3e71-406a-93ce-01a5576e3f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:27.428569 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865a76f-3e71-406a-93ce-01a5576e3f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:27.428933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865a76f-3e71-406a-93ce-01a5576e3f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:27.429722 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865a76f-3e71-406a-93ce-01a5576e3f55 admin admin] PATH_INFO: `/v3/projects/49609873841540098752bbcf10b0c6a2` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:27.435387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865a76f-3e71-406a-93ce-01a5576e3f55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49609873841540098752bbcf10b0c6a2)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:27.437839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865a76f-3e71-406a-93ce-01a5576e3f55 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:27.645213 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9865a76f-3e71-406a-93ce-01a5576e3f55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49609873841540098752bbcf10b0c6a2', 'request_id': 'req-9865a76f-3e71-406a-93ce-01a5576e3f55'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:27.657293 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:27.664257 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9865a76f-3e71-406a-93ce-01a5576e3f55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49609873841540098752bbcf10b0c6a2', 'request_id': 'req-9865a76f-3e71-406a-93ce-01a5576e3f55'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:27.802907 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9865a76f-3e71-406a-93ce-01a5576e3f55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49609873841540098752bbcf10b0c6a2', 'request_id': 'req-9865a76f-3e71-406a-93ce-01a5576e3f55'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:27.841887 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9865a76f-3e71-406a-93ce-01a5576e3f55 admin admin] The token cache is being invalidate because project 49609873841540098752bbcf10b0c6a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:35:27.842421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9865a76f-3e71-406a-93ce-01a5576e3f55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:27.887233 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9865a76f-3e71-406a-93ce-01a5576e3f55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:27.898143 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1068/2127] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:35:27 2022] DELETE /identity/v3/projects/49609873841540098752bbcf10b0c6a2 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:27.916136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c99b8e-68c3-4aa0-a539-7b3a050ce80f service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:27.988830 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c99b8e-68c3-4aa0-a539-7b3a050ce80f service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:28.152900 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a41a3c-66a4-4723-9e61-2f5c8abf5b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:28.154393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a41a3c-66a4-4723-9e61-2f5c8abf5b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:28.154902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a41a3c-66a4-4723-9e61-2f5c8abf5b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:28.155420 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a41a3c-66a4-4723-9e61-2f5c8abf5b59 admin admin] PATH_INFO: `/v3/users/e791862730944cb2aac31d261d1dec31` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:28.157552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a41a3c-66a4-4723-9e61-2f5c8abf5b59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e791862730944cb2aac31d261d1dec31)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:28.160922 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a41a3c-66a4-4723-9e61-2f5c8abf5b59 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:28.611953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d0a41a3c-66a4-4723-9e61-2f5c8abf5b59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e791862730944cb2aac31d261d1dec31'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:28.646204 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d0a41a3c-66a4-4723-9e61-2f5c8abf5b59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e791862730944cb2aac31d261d1dec31'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:28.649102 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d0a41a3c-66a4-4723-9e61-2f5c8abf5b59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e791862730944cb2aac31d261d1dec31'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:28.670912 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d0a41a3c-66a4-4723-9e61-2f5c8abf5b59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e791862730944cb2aac31d261d1dec31'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:28.672726 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d0a41a3c-66a4-4723-9e61-2f5c8abf5b59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e791862730944cb2aac31d261d1dec31'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:28.859392 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1060/2128] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:35:27 2022] DELETE /identity/v3/users/e791862730944cb2aac31d261d1dec31 => generated 0 bytes in 957 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:30.687794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865a76f-3e71-406a-93ce-01a5576e3f55 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:30.690178 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9865a76f-3e71-406a-93ce-01a5576e3f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:30.763367 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865a76f-3e71-406a-93ce-01a5576e3f55 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:30.863739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a41a3c-66a4-4723-9e61-2f5c8abf5b59 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:30.869108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d0a41a3c-66a4-4723-9e61-2f5c8abf5b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:31.150499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10186659-6e58-4e94-bf1c-f3ed4296fcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:31.152172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10186659-6e58-4e94-bf1c-f3ed4296fcfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:31.152711 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10186659-6e58-4e94-bf1c-f3ed4296fcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:31.153323 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10186659-6e58-4e94-bf1c-f3ed4296fcfa admin admin] PATH_INFO: `/v3/projects/746f576fcab94636bf74991859cefaad` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:31.157527 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10186659-6e58-4e94-bf1c-f3ed4296fcfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=746f576fcab94636bf74991859cefaad)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:31.161662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10186659-6e58-4e94-bf1c-f3ed4296fcfa admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:31.332477 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a41a3c-66a4-4723-9e61-2f5c8abf5b59 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:31.361193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68137b45-7714-4cca-9c37-5653660c56fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bee90bf951f04c2e8e19ceeacb8cfcb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:31.372683 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68137b45-7714-4cca-9c37-5653660c56fe service placement] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:31.375361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68137b45-7714-4cca-9c37-5653660c56fe service placement] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:31.376539 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68137b45-7714-4cca-9c37-5653660c56fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:31.389332 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-68137b45-7714-4cca-9c37-5653660c56fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:31.444963 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68137b45-7714-4cca-9c37-5653660c56fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:31.447072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68137b45-7714-4cca-9c37-5653660c56fe service placement] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:31.495232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1061/2129] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:35:31.499243 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-10186659-6e58-4e94-bf1c-f3ed4296fcfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '746f576fcab94636bf74991859cefaad', 'request_id': 'req-10186659-6e58-4e94-bf1c-f3ed4296fcfa'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:31.515965 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-10186659-6e58-4e94-bf1c-f3ed4296fcfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '746f576fcab94636bf74991859cefaad', 'request_id': 'req-10186659-6e58-4e94-bf1c-f3ed4296fcfa'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:31.926548 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-10186659-6e58-4e94-bf1c-f3ed4296fcfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '746f576fcab94636bf74991859cefaad', 'request_id': 'req-10186659-6e58-4e94-bf1c-f3ed4296fcfa'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:31.983962 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-10186659-6e58-4e94-bf1c-f3ed4296fcfa admin admin] The token cache is being invalidate because project 746f576fcab94636bf74991859cefaad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:35:31.986129 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-10186659-6e58-4e94-bf1c-f3ed4296fcfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:31.987779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-10186659-6e58-4e94-bf1c-f3ed4296fcfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:31.993371 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1069/2130] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:35:30 2022] DELETE /identity/v3/projects/746f576fcab94636bf74991859cefaad => generated 0 bytes in 1307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:32.036411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d35f8d-4125-4d5d-9f0c-5f333e485ff3 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:32.036411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d35f8d-4125-4d5d-9f0c-5f333e485ff3 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:32.036411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d35f8d-4125-4d5d-9f0c-5f333e485ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:32.036411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:32.044959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-b0d35f8d-4125-4d5d-9f0c-5f333e485ff3 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:32.046400 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b0d35f8d-4125-4d5d-9f0c-5f333e485ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:32.312793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1062/2131] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:32.323689 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a38cc-9fb0-4654-9f34-1503162e9f07 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:32.324502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a38cc-9fb0-4654-9f34-1503162e9f07 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:32.325184 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a38cc-9fb0-4654-9f34-1503162e9f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:32.422634 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-5a7a38cc-9fb0-4654-9f34-1503162e9f07 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:32.424940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-5a7a38cc-9fb0-4654-9f34-1503162e9f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:32.442015 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1070/2132] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:32.451207 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d35f8d-4125-4d5d-9f0c-5f333e485ff3 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:32.527603 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d35f8d-4125-4d5d-9f0c-5f333e485ff3 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:32.556469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9ba7b-37f0-481d-8ddf-cd20f60d2150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:32.560124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9ba7b-37f0-481d-8ddf-cd20f60d2150 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:32.561007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9ba7b-37f0-481d-8ddf-cd20f60d2150 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:32.561507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9ba7b-37f0-481d-8ddf-cd20f60d2150 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:32.563829 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9ba7b-37f0-481d-8ddf-cd20f60d2150 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:35:32.565224 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9ba7b-37f0-481d-8ddf-cd20f60d2150 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:32.568638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9ba7b-37f0-481d-8ddf-cd20f60d2150 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:32.667745 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1063/2133] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:35:32.673318 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a38cc-9fb0-4654-9f34-1503162e9f07 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:32.692372 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a38cc-9fb0-4654-9f34-1503162e9f07 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:32.723825 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e095e-5595-40ca-ab94-90b54dc4dca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:32.728845 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e095e-5595-40ca-ab94-90b54dc4dca0 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:32.729128 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e095e-5595-40ca-ab94-90b54dc4dca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:32.732487 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e095e-5595-40ca-ab94-90b54dc4dca0 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:32.735700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e095e-5595-40ca-ab94-90b54dc4dca0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:32.746606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e095e-5595-40ca-ab94-90b54dc4dca0 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:32.914605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1071/2134] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:35:32 2022] POST /identity/v3/projects => generated 379 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:35:32.921243 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9ba7b-37f0-481d-8ddf-cd20f60d2150 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:32.955816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9ba7b-37f0-481d-8ddf-cd20f60d2150 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:33.212240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c03be1-c728-4f35-a521-18a042e0528c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:33.217578 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c03be1-c728-4f35-a521-18a042e0528c admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:33.218573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c03be1-c728-4f35-a521-18a042e0528c admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:33.219608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c03be1-c728-4f35-a521-18a042e0528c admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:33.225062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c03be1-c728-4f35-a521-18a042e0528c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:33.236487 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c03be1-c728-4f35-a521-18a042e0528c admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:33.460551 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1064/2135] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:35:32 2022] POST /identity/v3/users => generated 290 bytes in 540 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:35:33.467122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e095e-5595-40ca-ab94-90b54dc4dca0 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:33.557000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e095e-5595-40ca-ab94-90b54dc4dca0 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:33.577695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580e840-9fc5-42dc-a900-3816113a1b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:33.578810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580e840-9fc5-42dc-a900-3816113a1b6c admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:33.579291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580e840-9fc5-42dc-a900-3816113a1b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:33.579754 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580e840-9fc5-42dc-a900-3816113a1b6c admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:33.593390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580e840-9fc5-42dc-a900-3816113a1b6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:33.593390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580e840-9fc5-42dc-a900-3816113a1b6c admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:33.664503 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c03be1-c728-4f35-a521-18a042e0528c admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:33.679192 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1072/2136] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:35:33.709279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c03be1-c728-4f35-a521-18a042e0528c admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:33.732203 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd624c16-7b13-4877-8006-cefc20469faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:33.733265 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd624c16-7b13-4877-8006-cefc20469faa admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:33.733746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd624c16-7b13-4877-8006-cefc20469faa admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:33.734223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd624c16-7b13-4877-8006-cefc20469faa admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:33.735530 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd624c16-7b13-4877-8006-cefc20469faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:33.746647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd624c16-7b13-4877-8006-cefc20469faa admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:33.803464 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580e840-9fc5-42dc-a900-3816113a1b6c admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:33.804180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1065/2137] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:35:33.811489 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af416774-9216-4063-8ce2-bdfb9b9c691c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:33.812028 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af416774-9216-4063-8ce2-bdfb9b9c691c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:33.812581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af416774-9216-4063-8ce2-bdfb9b9c691c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:33.853932 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580e840-9fc5-42dc-a900-3816113a1b6c admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:33.903125 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-af416774-9216-4063-8ce2-bdfb9b9c691c None None] MFA Rules not processed for user `426ecc795ae243ffa7c7e1746e8fc4aa`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:33.905237 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-af416774-9216-4063-8ce2-bdfb9b9c691c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:33.910329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1160087-572d-4ae5-bce5-a4c66e2a06bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:33.911549 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1160087-572d-4ae5-bce5-a4c66e2a06bd admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:33.921451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1160087-572d-4ae5-bce5-a4c66e2a06bd admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:33.922141 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1160087-572d-4ae5-bce5-a4c66e2a06bd admin admin] PATH_INFO: `/v3/projects/b8f304005b5d4d8c84b0cd7561c07234/users/82e4b12e728546f0addcbd5633c40a8b/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:34.032074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1160087-572d-4ae5-bce5-a4c66e2a06bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8f304005b5d4d8c84b0cd7561c07234, user_id=82e4b12e728546f0addcbd5633c40a8b, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:34.037347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1160087-572d-4ae5-bce5-a4c66e2a06bd admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:34.120766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1066/2138] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:34.270676 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1073/2139] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:35:33 2022] PUT /identity/v3/projects/b8f304005b5d4d8c84b0cd7561c07234/users/82e4b12e728546f0addcbd5633c40a8b/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:34.285805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f059b-2336-49b3-87f7-343b7dccb35a None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:34.286629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f059b-2336-49b3-87f7-343b7dccb35a None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:34.286798 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f059b-2336-49b3-87f7-343b7dccb35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:34.347416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-245f059b-2336-49b3-87f7-343b7dccb35a None None] MFA Rules not processed for user `82e4b12e728546f0addcbd5633c40a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:34.362533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-245f059b-2336-49b3-87f7-343b7dccb35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:34.362533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:34.555390 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1067/2140] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:34.563195 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1160087-572d-4ae5-bce5-a4c66e2a06bd admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:34.587852 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:34.693747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1160087-572d-4ae5-bce5-a4c66e2a06bd admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:34.700986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b802e3-6444-4ff5-bb56-3734f6a4c0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:34.702015 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b802e3-6444-4ff5-bb56-3734f6a4c0bb admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:34.702721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b802e3-6444-4ff5-bb56-3734f6a4c0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:34.702918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b802e3-6444-4ff5-bb56-3734f6a4c0bb admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:34.704495 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b802e3-6444-4ff5-bb56-3734f6a4c0bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:34.709008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b802e3-6444-4ff5-bb56-3734f6a4c0bb admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:35.047677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1074/2141] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:35:34 2022] POST /identity/v3/projects => generated 381 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:35:35.053458 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f059b-2336-49b3-87f7-343b7dccb35a None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:35.103614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f059b-2336-49b3-87f7-343b7dccb35a None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:35.319306 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f906d1-1bc9-4e3e-abf9-fb71d3d80d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:35.324892 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f906d1-1bc9-4e3e-abf9-fb71d3d80d7a admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:35.324892 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f906d1-1bc9-4e3e-abf9-fb71d3d80d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:35.324892 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f906d1-1bc9-4e3e-abf9-fb71d3d80d7a admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:35.324892 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f906d1-1bc9-4e3e-abf9-fb71d3d80d7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:35.328279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f906d1-1bc9-4e3e-abf9-fb71d3d80d7a admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:35.539045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1068/2142] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:35:35 2022] POST /identity/v3/users => generated 297 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:35:35.546354 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b802e3-6444-4ff5-bb56-3734f6a4c0bb admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:35.572636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b802e3-6444-4ff5-bb56-3734f6a4c0bb admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:35.606126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ba8f1-e7b9-42de-a3eb-a3431cde07c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:35.607690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ba8f1-e7b9-42de-a3eb-a3431cde07c0 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:35.608243 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ba8f1-e7b9-42de-a3eb-a3431cde07c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:35.608653 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ba8f1-e7b9-42de-a3eb-a3431cde07c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:35.609810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ba8f1-e7b9-42de-a3eb-a3431cde07c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:35.613804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ba8f1-e7b9-42de-a3eb-a3431cde07c0 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:35.637803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1075/2143] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:35:35.639885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f906d1-1bc9-4e3e-abf9-fb71d3d80d7a admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:35.669727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f906d1-1bc9-4e3e-abf9-fb71d3d80d7a admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:35.680179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19151a0-cc0e-44f3-8217-4bd0c4828127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:35.681752 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19151a0-cc0e-44f3-8217-4bd0c4828127 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:35.682244 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19151a0-cc0e-44f3-8217-4bd0c4828127 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:35.683309 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19151a0-cc0e-44f3-8217-4bd0c4828127 admin admin] PATH_INFO: `/v3/projects/83b2757f91574534963ef004fe8fa3a6/users/a1ff44b7a31048cc84ab1af2340202a0/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:35.713700 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19151a0-cc0e-44f3-8217-4bd0c4828127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83b2757f91574534963ef004fe8fa3a6, user_id=a1ff44b7a31048cc84ab1af2340202a0, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:35.720356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19151a0-cc0e-44f3-8217-4bd0c4828127 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:35.959900 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1069/2144] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:35:35 2022] PUT /identity/v3/projects/83b2757f91574534963ef004fe8fa3a6/users/a1ff44b7a31048cc84ab1af2340202a0/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:35.964027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ba8f1-e7b9-42de-a3eb-a3431cde07c0 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:35.992167 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ba8f1-e7b9-42de-a3eb-a3431cde07c0 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:36.137309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd44cf-062d-432d-8d9e-51b33840ee80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:36.138726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd44cf-062d-432d-8d9e-51b33840ee80 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:36.139549 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd44cf-062d-432d-8d9e-51b33840ee80 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:36.144026 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd44cf-062d-432d-8d9e-51b33840ee80 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:36.144026 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd44cf-062d-432d-8d9e-51b33840ee80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:36.145844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd44cf-062d-432d-8d9e-51b33840ee80 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:36.187692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1076/2145] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:35:36.191916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19151a0-cc0e-44f3-8217-4bd0c4828127 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:36.218798 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19151a0-cc0e-44f3-8217-4bd0c4828127 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:36.228604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc0e9f-9f71-40fa-a5d1-8de863491389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:36.229449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc0e9f-9f71-40fa-a5d1-8de863491389 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:36.229780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc0e9f-9f71-40fa-a5d1-8de863491389 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:36.230078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc0e9f-9f71-40fa-a5d1-8de863491389 admin admin] PATH_INFO: `/v3/projects/83b2757f91574534963ef004fe8fa3a6/users/a1ff44b7a31048cc84ab1af2340202a0/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:36.236302 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc0e9f-9f71-40fa-a5d1-8de863491389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83b2757f91574534963ef004fe8fa3a6, user_id=a1ff44b7a31048cc84ab1af2340202a0, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:36.244076 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc0e9f-9f71-40fa-a5d1-8de863491389 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:36.287340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1070/2146] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:35:36 2022] PUT /identity/v3/projects/83b2757f91574534963ef004fe8fa3a6/users/a1ff44b7a31048cc84ab1af2340202a0/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:36.317244 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffa8b4-fe9d-421d-a7a2-61a2ee1a295d None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:36.321846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffa8b4-fe9d-421d-a7a2-61a2ee1a295d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:36.322644 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffa8b4-fe9d-421d-a7a2-61a2ee1a295d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:36.432152 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-85ffa8b4-fe9d-421d-a7a2-61a2ee1a295d None None] MFA Rules not processed for user `a1ff44b7a31048cc84ab1af2340202a0`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:36.434798 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-85ffa8b4-fe9d-421d-a7a2-61a2ee1a295d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:36.669482 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1077/2147] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:36.706459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc0e9f-9f71-40fa-a5d1-8de863491389 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:36.709171 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-94cc0e9f-9f71-40fa-a5d1-8de863491389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:36.892157 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc0e9f-9f71-40fa-a5d1-8de863491389 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:36.921456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b8291-5d5f-48ec-adc2-11fb4396ef4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:36.922636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b8291-5d5f-48ec-adc2-11fb4396ef4f service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:36.926021 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b8291-5d5f-48ec-adc2-11fb4396ef4f service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:36.926630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b8291-5d5f-48ec-adc2-11fb4396ef4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:36.975552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b8291-5d5f-48ec-adc2-11fb4396ef4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:36.988737 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b8291-5d5f-48ec-adc2-11fb4396ef4f service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:37.010225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:37.054719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1071/2148] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:36 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:35:38.820252 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ffa8b4-fe9d-421d-a7a2-61a2ee1a295d None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:38.833428 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:38.851361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ffa8b4-fe9d-421d-a7a2-61a2ee1a295d None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:38.857371 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2c958-1dd8-454a-bff1-ea7cdf45d601 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ff44b7a31048cc84ab1af2340202a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83b2757f91574534963ef004fe8fa3a6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:38.858348 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2c958-1dd8-454a-bff1-ea7cdf45d601 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:38.858850 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2c958-1dd8-454a-bff1-ea7cdf45d601 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:38.859362 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2c958-1dd8-454a-bff1-ea7cdf45d601 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:38.860394 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2c958-1dd8-454a-bff1-ea7cdf45d601 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:35:38.860910 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2c958-1dd8-454a-bff1-ea7cdf45d601 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:38.863089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2c958-1dd8-454a-bff1-ea7cdf45d601 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:38.879369 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1078/2149] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:35:38.885216 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b8291-5d5f-48ec-adc2-11fb4396ef4f service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:38.903385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b8291-5d5f-48ec-adc2-11fb4396ef4f service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:38.914075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ca693-a4b6-4bb6-a1a6-7ce1fc02e65f tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ff44b7a31048cc84ab1af2340202a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83b2757f91574534963ef004fe8fa3a6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:38.915405 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ca693-a4b6-4bb6-a1a6-7ce1fc02e65f tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:38.915909 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ca693-a4b6-4bb6-a1a6-7ce1fc02e65f tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:38.916381 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ca693-a4b6-4bb6-a1a6-7ce1fc02e65f tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:38.917837 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ca693-a4b6-4bb6-a1a6-7ce1fc02e65f tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:38.920397 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ca693-a4b6-4bb6-a1a6-7ce1fc02e65f tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:39.069014 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1072/2150] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:35:38 2022] POST /identity/v3/projects => generated 354 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:35:39.074594 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2c958-1dd8-454a-bff1-ea7cdf45d601 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:39.099548 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2c958-1dd8-454a-bff1-ea7cdf45d601 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:39.372635 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880889e1-2bb4-41aa-a798-68722be3dad8 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ff44b7a31048cc84ab1af2340202a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83b2757f91574534963ef004fe8fa3a6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:39.373286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880889e1-2bb4-41aa-a798-68722be3dad8 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:39.373592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880889e1-2bb4-41aa-a798-68722be3dad8 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:39.373956 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880889e1-2bb4-41aa-a798-68722be3dad8 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:39.374731 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880889e1-2bb4-41aa-a798-68722be3dad8 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:35:39.375083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880889e1-2bb4-41aa-a798-68722be3dad8 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:39.378095 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880889e1-2bb4-41aa-a798-68722be3dad8 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:39.408205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1079/2151] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:35:39.503161 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ca693-a4b6-4bb6-a1a6-7ce1fc02e65f tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:39.567287 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ca693-a4b6-4bb6-a1a6-7ce1fc02e65f tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:39.602777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c897b-a3c5-4e7b-a085-6a82ef3c595b tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1ff44b7a31048cc84ab1af2340202a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83b2757f91574534963ef004fe8fa3a6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:39.602777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c897b-a3c5-4e7b-a085-6a82ef3c595b tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:39.602777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c897b-a3c5-4e7b-a085-6a82ef3c595b tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:39.605728 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c897b-a3c5-4e7b-a085-6a82ef3c595b tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] PATH_INFO: `/v3/projects/20ee8d10ec5349039018d7ccee644cb5` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:39.605728 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c897b-a3c5-4e7b-a085-6a82ef3c595b tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] RBAC: Authorizing `identity:delete_project(project_id=20ee8d10ec5349039018d7ccee644cb5)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:39.605728 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c897b-a3c5-4e7b-a085-6a82ef3c595b tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:41.114348 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-473c897b-a3c5-4e7b-a085-6a82ef3c595b tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20ee8d10ec5349039018d7ccee644cb5', 'request_id': 'req-473c897b-a3c5-4e7b-a085-6a82ef3c595b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:41.129978 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-473c897b-a3c5-4e7b-a085-6a82ef3c595b tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20ee8d10ec5349039018d7ccee644cb5', 'request_id': 'req-473c897b-a3c5-4e7b-a085-6a82ef3c595b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:41.338878 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-473c897b-a3c5-4e7b-a085-6a82ef3c595b tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20ee8d10ec5349039018d7ccee644cb5', 'request_id': 'req-473c897b-a3c5-4e7b-a085-6a82ef3c595b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:41.377842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-473c897b-a3c5-4e7b-a085-6a82ef3c595b tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] The token cache is being invalidate because project 20ee8d10ec5349039018d7ccee644cb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:35:41.378263 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-473c897b-a3c5-4e7b-a085-6a82ef3c595b tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:41.380482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-473c897b-a3c5-4e7b-a085-6a82ef3c595b tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:41.399536 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1073/2152] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:35:39 2022] DELETE /identity/v3/projects/20ee8d10ec5349039018d7ccee644cb5 => generated 0 bytes in 1891 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:42.562050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880889e1-2bb4-41aa-a798-68722be3dad8 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:42.576651 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-880889e1-2bb4-41aa-a798-68722be3dad8 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:42.774450 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880889e1-2bb4-41aa-a798-68722be3dad8 tempest-VolumeQuotaClassesTest-1441347004 tempest-VolumeQuotaClassesTest-1441347004-project-admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:42.927977 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbeda48-90ee-4b77-beaa-bcf126aa2bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:42.931086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeda48-90ee-4b77-beaa-bcf126aa2bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:42.931086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeda48-90ee-4b77-beaa-bcf126aa2bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:42.931086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeda48-90ee-4b77-beaa-bcf126aa2bc7 admin admin] PATH_INFO: `/v3/users/82e4b12e728546f0addcbd5633c40a8b` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:42.933010 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbeda48-90ee-4b77-beaa-bcf126aa2bc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82e4b12e728546f0addcbd5633c40a8b)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:42.936842 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbeda48-90ee-4b77-beaa-bcf126aa2bc7 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:43.331209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:43.367488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0cbeda48-90ee-4b77-beaa-bcf126aa2bc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82e4b12e728546f0addcbd5633c40a8b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:43.480491 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0cbeda48-90ee-4b77-beaa-bcf126aa2bc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82e4b12e728546f0addcbd5633c40a8b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:43.482006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0cbeda48-90ee-4b77-beaa-bcf126aa2bc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82e4b12e728546f0addcbd5633c40a8b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:43.482993 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0cbeda48-90ee-4b77-beaa-bcf126aa2bc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82e4b12e728546f0addcbd5633c40a8b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:43.508659 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0cbeda48-90ee-4b77-beaa-bcf126aa2bc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82e4b12e728546f0addcbd5633c40a8b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:43.561945 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1080/2153] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:35:42 2022] DELETE /identity/v3/users/82e4b12e728546f0addcbd5633c40a8b => generated 0 bytes in 1001 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:43.567961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca54a2-112e-4053-b1f1-d0b9ad4aabb0 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:43.568501 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca54a2-112e-4053-b1f1-d0b9ad4aabb0 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:43.568501 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca54a2-112e-4053-b1f1-d0b9ad4aabb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:43.598082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-56ca54a2-112e-4053-b1f1-d0b9ad4aabb0 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:43.599462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-56ca54a2-112e-4053-b1f1-d0b9ad4aabb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:43.774712 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1074/2154] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:43.801259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbeda48-90ee-4b77-beaa-bcf126aa2bc7 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:43.805676 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0cbeda48-90ee-4b77-beaa-bcf126aa2bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:43.937210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbeda48-90ee-4b77-beaa-bcf126aa2bc7 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:43.941255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9d902-4b7c-44c4-9a9d-9baf9220ba66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:43.941900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9d902-4b7c-44c4-9a9d-9baf9220ba66 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:43.942388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9d902-4b7c-44c4-9a9d-9baf9220ba66 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:43.942684 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9d902-4b7c-44c4-9a9d-9baf9220ba66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:43.955038 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9d902-4b7c-44c4-9a9d-9baf9220ba66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:43.955829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9d902-4b7c-44c4-9a9d-9baf9220ba66 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:43.988177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1081/2155] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:35:44.748986 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca54a2-112e-4053-b1f1-d0b9ad4aabb0 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:44.751902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-56ca54a2-112e-4053-b1f1-d0b9ad4aabb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:44.790479 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca54a2-112e-4053-b1f1-d0b9ad4aabb0 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:44.797332 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:44.798416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:44.803460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:44.803460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b admin admin] PATH_INFO: `/v3/projects/b8f304005b5d4d8c84b0cd7561c07234` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:44.803460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8f304005b5d4d8c84b0cd7561c07234)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:44.805851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:45.011514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8f304005b5d4d8c84b0cd7561c07234', 'request_id': 'req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:45.025702 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8f304005b5d4d8c84b0cd7561c07234', 'request_id': 'req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:45.034968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:45.174530 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8f304005b5d4d8c84b0cd7561c07234', 'request_id': 'req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:45.190813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b admin admin] The token cache is being invalidate because project b8f304005b5d4d8c84b0cd7561c07234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:35:45.191580 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:45.192674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:45.195370 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1075/2156] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:35:44 2022] DELETE /identity/v3/projects/b8f304005b5d4d8c84b0cd7561c07234 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:45.206519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9d902-4b7c-44c4-9a9d-9baf9220ba66 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:45.231491 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9d902-4b7c-44c4-9a9d-9baf9220ba66 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:45.348692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41ba49-f796-4b7d-a1ce-849307ef9a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:45.349490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41ba49-f796-4b7d-a1ce-849307ef9a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:45.349953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41ba49-f796-4b7d-a1ce-849307ef9a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:45.351317 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41ba49-f796-4b7d-a1ce-849307ef9a7c admin admin] PATH_INFO: `/v3/users/a1ff44b7a31048cc84ab1af2340202a0` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:45.353816 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41ba49-f796-4b7d-a1ce-849307ef9a7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1ff44b7a31048cc84ab1af2340202a0)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:45.356811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41ba49-f796-4b7d-a1ce-849307ef9a7c admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:45.664697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-bf41ba49-f796-4b7d-a1ce-849307ef9a7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1ff44b7a31048cc84ab1af2340202a0'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:45.834629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-bf41ba49-f796-4b7d-a1ce-849307ef9a7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1ff44b7a31048cc84ab1af2340202a0'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:45.839574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-bf41ba49-f796-4b7d-a1ce-849307ef9a7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1ff44b7a31048cc84ab1af2340202a0'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:45.839574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-bf41ba49-f796-4b7d-a1ce-849307ef9a7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1ff44b7a31048cc84ab1af2340202a0'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:45.839574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:45.852450 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-bf41ba49-f796-4b7d-a1ce-849307ef9a7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1ff44b7a31048cc84ab1af2340202a0'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:45.931105 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1082/2157] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:35:45 2022] DELETE /identity/v3/users/a1ff44b7a31048cc84ab1af2340202a0 => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:47.123774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:47.126559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:47.141343 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a72ed-fc6f-4ec1-ada0-9c1eaf7f197b admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:47.251176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9663a2-4f66-45f9-9c8e-c8c24dddadfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:47.251911 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9663a2-4f66-45f9-9c8e-c8c24dddadfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:47.252440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9663a2-4f66-45f9-9c8e-c8c24dddadfd admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:47.252814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9663a2-4f66-45f9-9c8e-c8c24dddadfd admin admin] PATH_INFO: `/v3/projects/83b2757f91574534963ef004fe8fa3a6` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:47.254289 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9663a2-4f66-45f9-9c8e-c8c24dddadfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=83b2757f91574534963ef004fe8fa3a6)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:47.257033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9663a2-4f66-45f9-9c8e-c8c24dddadfd admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:47.441266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-4f9663a2-4f66-45f9-9c8e-c8c24dddadfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83b2757f91574534963ef004fe8fa3a6', 'request_id': 'req-4f9663a2-4f66-45f9-9c8e-c8c24dddadfd'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:47.451388 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-4f9663a2-4f66-45f9-9c8e-c8c24dddadfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83b2757f91574534963ef004fe8fa3a6', 'request_id': 'req-4f9663a2-4f66-45f9-9c8e-c8c24dddadfd'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:47.530360 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-4f9663a2-4f66-45f9-9c8e-c8c24dddadfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83b2757f91574534963ef004fe8fa3a6', 'request_id': 'req-4f9663a2-4f66-45f9-9c8e-c8c24dddadfd'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:47.554084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-4f9663a2-4f66-45f9-9c8e-c8c24dddadfd admin admin] The token cache is being invalidate because project 83b2757f91574534963ef004fe8fa3a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:35:47.554831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-4f9663a2-4f66-45f9-9c8e-c8c24dddadfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:47.556192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-4f9663a2-4f66-45f9-9c8e-c8c24dddadfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:47.559052 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1076/2158] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:35:47 2022] DELETE /identity/v3/projects/83b2757f91574534963ef004fe8fa3a6 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:47.566861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478ccc7-3b03-4ebc-9f64-056318b95370 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:47.567653 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478ccc7-3b03-4ebc-9f64-056318b95370 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:47.567844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478ccc7-3b03-4ebc-9f64-056318b95370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:47.599499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-6478ccc7-3b03-4ebc-9f64-056318b95370 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:47.599499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-6478ccc7-3b03-4ebc-9f64-056318b95370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:47.700349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1083/2159] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:47.713616 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27514884-2d01-46e7-96b3-5dc35203d2ac None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:47.714005 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27514884-2d01-46e7-96b3-5dc35203d2ac None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:47.714496 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27514884-2d01-46e7-96b3-5dc35203d2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:47.748136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-27514884-2d01-46e7-96b3-5dc35203d2ac None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:47.750063 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-27514884-2d01-46e7-96b3-5dc35203d2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:47.767154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1077/2160] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:47.773789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478ccc7-3b03-4ebc-9f64-056318b95370 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:47.795065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478ccc7-3b03-4ebc-9f64-056318b95370 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:47.810833 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752955d0-b697-43bc-8506-0d27769edd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:47.812041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752955d0-b697-43bc-8506-0d27769edd83 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:47.812645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752955d0-b697-43bc-8506-0d27769edd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:47.813308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752955d0-b697-43bc-8506-0d27769edd83 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:47.814344 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752955d0-b697-43bc-8506-0d27769edd83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:35:47.814718 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752955d0-b697-43bc-8506-0d27769edd83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:47.816858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752955d0-b697-43bc-8506-0d27769edd83 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:47.839258 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1084/2161] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:35:47.845972 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27514884-2d01-46e7-96b3-5dc35203d2ac None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:47.858754 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27514884-2d01-46e7-96b3-5dc35203d2ac None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:47.868827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74835e-ffd7-4d7f-b28c-f4e659453146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:47.869543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74835e-ffd7-4d7f-b28c-f4e659453146 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:47.869832 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74835e-ffd7-4d7f-b28c-f4e659453146 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:47.870237 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74835e-ffd7-4d7f-b28c-f4e659453146 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:47.870922 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b74835e-ffd7-4d7f-b28c-f4e659453146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:47.873463 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b74835e-ffd7-4d7f-b28c-f4e659453146 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:47.977706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1078/2162] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:35:47 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:35:47.986656 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752955d0-b697-43bc-8506-0d27769edd83 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:48.005240 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752955d0-b697-43bc-8506-0d27769edd83 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:48.167819 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f72635-f69e-426f-9f2e-1751d97bdeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:48.168917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f72635-f69e-426f-9f2e-1751d97bdeef admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:48.169402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f72635-f69e-426f-9f2e-1751d97bdeef admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:48.169922 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f72635-f69e-426f-9f2e-1751d97bdeef admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:48.171210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f72635-f69e-426f-9f2e-1751d97bdeef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:48.175154 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f72635-f69e-426f-9f2e-1751d97bdeef admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:48.322910 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1085/2163] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:35:47 2022] POST /identity/v3/users => generated 294 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:35:48.329502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74835e-ffd7-4d7f-b28c-f4e659453146 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:48.354614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74835e-ffd7-4d7f-b28c-f4e659453146 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:48.367818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb574a03-ce02-4ca9-a339-9431894915f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:48.369290 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb574a03-ce02-4ca9-a339-9431894915f7 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:48.370390 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb574a03-ce02-4ca9-a339-9431894915f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:48.371207 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb574a03-ce02-4ca9-a339-9431894915f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:48.372593 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb574a03-ce02-4ca9-a339-9431894915f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:48.375444 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb574a03-ce02-4ca9-a339-9431894915f7 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:48.393169 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1079/2164] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:35:48.400329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f72635-f69e-426f-9f2e-1751d97bdeef admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:48.432585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f72635-f69e-426f-9f2e-1751d97bdeef admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:48.441459 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f871fe-6537-449f-904f-4a2307d858ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:48.442543 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f871fe-6537-449f-904f-4a2307d858ee admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:48.446350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f871fe-6537-449f-904f-4a2307d858ee admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:48.446350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f871fe-6537-449f-904f-4a2307d858ee admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:48.446350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f871fe-6537-449f-904f-4a2307d858ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:48.448496 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f871fe-6537-449f-904f-4a2307d858ee admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:48.469476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1086/2165] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:35:48.478162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb574a03-ce02-4ca9-a339-9431894915f7 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:48.498848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb574a03-ce02-4ca9-a339-9431894915f7 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:48.505775 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd17d00-9eb7-42fc-9764-2c0218323bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:48.507193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd17d00-9eb7-42fc-9764-2c0218323bb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:48.507697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd17d00-9eb7-42fc-9764-2c0218323bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:48.508126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd17d00-9eb7-42fc-9764-2c0218323bb7 admin admin] PATH_INFO: `/v3/projects/18372dab3b5249d5ac6ea115d41f6973/users/954226b6c5ea486da3ff886ede221c67/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:48.540344 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd17d00-9eb7-42fc-9764-2c0218323bb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18372dab3b5249d5ac6ea115d41f6973, user_id=954226b6c5ea486da3ff886ede221c67, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:48.546058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd17d00-9eb7-42fc-9764-2c0218323bb7 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:48.692580 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1080/2166] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:35:48 2022] PUT /identity/v3/projects/18372dab3b5249d5ac6ea115d41f6973/users/954226b6c5ea486da3ff886ede221c67/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:48.714679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8989f98-befc-4bb4-be3f-ba6e7c5a4b4b None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:48.715553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8989f98-befc-4bb4-be3f-ba6e7c5a4b4b None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:48.715707 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8989f98-befc-4bb4-be3f-ba6e7c5a4b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:48.772937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-d8989f98-befc-4bb4-be3f-ba6e7c5a4b4b None None] MFA Rules not processed for user `954226b6c5ea486da3ff886ede221c67`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:48.776390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d8989f98-befc-4bb4-be3f-ba6e7c5a4b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:48.888454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:48.960075 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1087/2167] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:48.965454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd17d00-9eb7-42fc-9764-2c0218323bb7 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:48.995351 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:49.091581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd17d00-9eb7-42fc-9764-2c0218323bb7 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:49.101138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefca871-aa69-4b25-880a-877ccd8b61fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:49.102012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefca871-aa69-4b25-880a-877ccd8b61fc admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:49.102388 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefca871-aa69-4b25-880a-877ccd8b61fc admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:49.102818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefca871-aa69-4b25-880a-877ccd8b61fc admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:49.104098 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefca871-aa69-4b25-880a-877ccd8b61fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:49.107802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefca871-aa69-4b25-880a-877ccd8b61fc admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:49.244547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1081/2168] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:35:48 2022] POST /identity/v3/projects => generated 385 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:35:49.249242 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8989f98-befc-4bb4-be3f-ba6e7c5a4b4b None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:49.268289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8989f98-befc-4bb4-be3f-ba6e7c5a4b4b None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:49.365515 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed92e1e-bba5-4990-b78e-b76982df8640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:49.366536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed92e1e-bba5-4990-b78e-b76982df8640 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:49.367925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed92e1e-bba5-4990-b78e-b76982df8640 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:49.367925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed92e1e-bba5-4990-b78e-b76982df8640 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:49.368610 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed92e1e-bba5-4990-b78e-b76982df8640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:49.371134 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed92e1e-bba5-4990-b78e-b76982df8640 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:49.557379 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1088/2169] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:35:49 2022] POST /identity/v3/users => generated 299 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:35:49.564340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefca871-aa69-4b25-880a-877ccd8b61fc admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:49.586342 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefca871-aa69-4b25-880a-877ccd8b61fc admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:49.596190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9b4da-6c51-4df1-b9eb-4f0c57ac20b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:49.599739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9b4da-6c51-4df1-b9eb-4f0c57ac20b3 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:49.599739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9b4da-6c51-4df1-b9eb-4f0c57ac20b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:49.599739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9b4da-6c51-4df1-b9eb-4f0c57ac20b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:49.601499 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9b4da-6c51-4df1-b9eb-4f0c57ac20b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:49.604684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9b4da-6c51-4df1-b9eb-4f0c57ac20b3 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:49.619439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1082/2170] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:35:49.632717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed92e1e-bba5-4990-b78e-b76982df8640 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:49.658755 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed92e1e-bba5-4990-b78e-b76982df8640 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:49.678428 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717e352-3d5f-47e2-a237-eef6ea4ac2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:49.680057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717e352-3d5f-47e2-a237-eef6ea4ac2c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:49.683833 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717e352-3d5f-47e2-a237-eef6ea4ac2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:49.684422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717e352-3d5f-47e2-a237-eef6ea4ac2c5 admin admin] PATH_INFO: `/v3/projects/9c44f21a6b424db2b29fe471713c7e65/users/8f3b5ca828b94226a1df530725a21f0c/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:49.712281 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717e352-3d5f-47e2-a237-eef6ea4ac2c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c44f21a6b424db2b29fe471713c7e65, user_id=8f3b5ca828b94226a1df530725a21f0c, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:49.715883 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717e352-3d5f-47e2-a237-eef6ea4ac2c5 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:49.836571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1089/2171] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:35:49 2022] PUT /identity/v3/projects/9c44f21a6b424db2b29fe471713c7e65/users/8f3b5ca828b94226a1df530725a21f0c/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:49.847866 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9b4da-6c51-4df1-b9eb-4f0c57ac20b3 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:49.886790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9b4da-6c51-4df1-b9eb-4f0c57ac20b3 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:50.063851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc99b64-cea3-4891-a1f7-4692fb386b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:50.065363 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc99b64-cea3-4891-a1f7-4692fb386b50 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:50.065858 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc99b64-cea3-4891-a1f7-4692fb386b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:50.066324 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc99b64-cea3-4891-a1f7-4692fb386b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:50.068155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc99b64-cea3-4891-a1f7-4692fb386b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:50.073289 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc99b64-cea3-4891-a1f7-4692fb386b50 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:50.091578 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1083/2172] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:35:50.099105 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717e352-3d5f-47e2-a237-eef6ea4ac2c5 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:50.137282 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717e352-3d5f-47e2-a237-eef6ea4ac2c5 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:50.150405 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d129d96-f8f6-4fa4-a0f1-cdce53e43679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:50.154160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d129d96-f8f6-4fa4-a0f1-cdce53e43679 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:50.154160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d129d96-f8f6-4fa4-a0f1-cdce53e43679 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:50.154160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d129d96-f8f6-4fa4-a0f1-cdce53e43679 admin admin] PATH_INFO: `/v3/projects/9c44f21a6b424db2b29fe471713c7e65/users/8f3b5ca828b94226a1df530725a21f0c/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:50.159319 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d129d96-f8f6-4fa4-a0f1-cdce53e43679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c44f21a6b424db2b29fe471713c7e65, user_id=8f3b5ca828b94226a1df530725a21f0c, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:50.167296 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d129d96-f8f6-4fa4-a0f1-cdce53e43679 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:50.187327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1090/2173] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:35:50 2022] PUT /identity/v3/projects/9c44f21a6b424db2b29fe471713c7e65/users/8f3b5ca828b94226a1df530725a21f0c/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:50.206454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a4f0e-ed16-4695-9177-efd5395914dd None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:50.206946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a4f0e-ed16-4695-9177-efd5395914dd None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:50.207496 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a4f0e-ed16-4695-9177-efd5395914dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:50.260709 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-898a4f0e-ed16-4695-9177-efd5395914dd None None] MFA Rules not processed for user `8f3b5ca828b94226a1df530725a21f0c`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:50.263254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-898a4f0e-ed16-4695-9177-efd5395914dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:50.434012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1084/2174] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:50.464008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d129d96-f8f6-4fa4-a0f1-cdce53e43679 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:50.468622 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-5d129d96-f8f6-4fa4-a0f1-cdce53e43679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:50.690396 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d129d96-f8f6-4fa4-a0f1-cdce53e43679 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:50.699112 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4497e-5acf-4284-ad13-2572884fec25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:50.700174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4497e-5acf-4284-ad13-2572884fec25 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:50.700665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4497e-5acf-4284-ad13-2572884fec25 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:50.701146 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4497e-5acf-4284-ad13-2572884fec25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:50.721831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4497e-5acf-4284-ad13-2572884fec25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:50.726556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4497e-5acf-4284-ad13-2572884fec25 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:50.746499 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1091/2175] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:35:53.591328 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe13517-bd03-4e81-8349-be0465209c2b None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:53.592026 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe13517-bd03-4e81-8349-be0465209c2b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:53.592026 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe13517-bd03-4e81-8349-be0465209c2b None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:53.595589 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1085/2176] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 7 13:35:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:35:53.605638 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e9028-1acc-416c-956f-0f39d1078d18 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:53.606772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e9028-1acc-416c-956f-0f39d1078d18 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:53.607321 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e9028-1acc-416c-956f-0f39d1078d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:53.637436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-a95e9028-1acc-416c-956f-0f39d1078d18 None None] MFA Rules not processed for user `97c363046db444cbb3d4346066f9ad42`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:53.639601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a95e9028-1acc-416c-956f-0f39d1078d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:53.665802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:53.762821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1092/2177] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 7 13:35:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:35:53.781691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe13517-bd03-4e81-8349-be0465209c2b None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:53.784381 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-bfe13517-bd03-4e81-8349-be0465209c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:53.791673 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e9028-1acc-416c-956f-0f39d1078d18 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:53.799295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:53.910903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe13517-bd03-4e81-8349-be0465209c2b None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:53.917811 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24fa34-39e6-42c5-9bb2-75c81971606c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1a48805d95640489d7ce3333dd687b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:53.919409 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24fa34-39e6-42c5-9bb2-75c81971606c service glance] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:53.920225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24fa34-39e6-42c5-9bb2-75c81971606c service glance] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:53.920785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24fa34-39e6-42c5-9bb2-75c81971606c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:53.940149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e9028-1acc-416c-956f-0f39d1078d18 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:53.945053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24fa34-39e6-42c5-9bb2-75c81971606c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:53.949906 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1dc88-4ecb-4f64-9aa0-d2ad00b0f67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:53.950317 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24fa34-39e6-42c5-9bb2-75c81971606c service glance] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:53.950846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1dc88-4ecb-4f64-9aa0-d2ad00b0f67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:53.951450 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1dc88-4ecb-4f64-9aa0-d2ad00b0f67e admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:53.952437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1dc88-4ecb-4f64-9aa0-d2ad00b0f67e admin admin] PATH_INFO: `/v3/users/954226b6c5ea486da3ff886ede221c67` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:53.955057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1dc88-4ecb-4f64-9aa0-d2ad00b0f67e admin admin] RBAC: Authorizing `identity:delete_user(user_id=954226b6c5ea486da3ff886ede221c67)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:53.959393 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1dc88-4ecb-4f64-9aa0-d2ad00b0f67e admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:53.997434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1086/2178] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:35:54.249904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9bc1dc88-4ecb-4f64-9aa0-d2ad00b0f67e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '954226b6c5ea486da3ff886ede221c67'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:54.341090 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6e656-6a5a-419c-811f-a7ffc3d492ca None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:54.341662 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6e656-6a5a-419c-811f-a7ffc3d492ca None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:54.342047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6e656-6a5a-419c-811f-a7ffc3d492ca None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:54.345016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1087/2179] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 7 13:35:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:35:54.354758 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde5d85-7038-45f8-a0a4-a3ecdd71da28 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:54.355223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde5d85-7038-45f8-a0a4-a3ecdd71da28 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:54.355678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde5d85-7038-45f8-a0a4-a3ecdd71da28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:54.360212 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9bc1dc88-4ecb-4f64-9aa0-d2ad00b0f67e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '954226b6c5ea486da3ff886ede221c67'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:54.360892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9bc1dc88-4ecb-4f64-9aa0-d2ad00b0f67e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '954226b6c5ea486da3ff886ede221c67'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:54.361865 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9bc1dc88-4ecb-4f64-9aa0-d2ad00b0f67e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '954226b6c5ea486da3ff886ede221c67'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:54.388340 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9bc1dc88-4ecb-4f64-9aa0-d2ad00b0f67e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '954226b6c5ea486da3ff886ede221c67'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:54.405803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-edde5d85-7038-45f8-a0a4-a3ecdd71da28 None None] MFA Rules not processed for user `5acdbdb6f23b4f10bcbab0387ea67ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:54.407875 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-edde5d85-7038-45f8-a0a4-a3ecdd71da28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:54.427395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1093/2180] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:35:53 2022] DELETE /identity/v3/users/954226b6c5ea486da3ff886ede221c67 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:54.435398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acd6e5-66f3-4dce-bb6c-62ff5fbcbb8d None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:54.435907 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acd6e5-66f3-4dce-bb6c-62ff5fbcbb8d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:54.436877 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acd6e5-66f3-4dce-bb6c-62ff5fbcbb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:54.481393 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-e4acd6e5-66f3-4dce-bb6c-62ff5fbcbb8d None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:54.481393 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e4acd6e5-66f3-4dce-bb6c-62ff5fbcbb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:54.542453 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:54.557980 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1088/2181] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 7 13:35:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:54.585122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde5d85-7038-45f8-a0a4-a3ecdd71da28 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:54.597366 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-edde5d85-7038-45f8-a0a4-a3ecdd71da28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:54.613714 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1094/2182] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:35:54.617559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde5d85-7038-45f8-a0a4-a3ecdd71da28 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:54.627281 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acd6e5-66f3-4dce-bb6c-62ff5fbcbb8d None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:54.717147 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa0c18-28c1-4f39-ad09-be1201a41307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:54.718867 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa0c18-28c1-4f39-ad09-be1201a41307 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:54.719487 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa0c18-28c1-4f39-ad09-be1201a41307 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:54.719908 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa0c18-28c1-4f39-ad09-be1201a41307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:54.731348 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa0c18-28c1-4f39-ad09-be1201a41307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:54.731649 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acd6e5-66f3-4dce-bb6c-62ff5fbcbb8d None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:54.733295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa0c18-28c1-4f39-ad09-be1201a41307 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:54.736810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c72e8c-cd67-4555-a76f-f78181afa5d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:54.737532 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c72e8c-cd67-4555-a76f-f78181afa5d1 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:54.737971 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c72e8c-cd67-4555-a76f-f78181afa5d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:54.738330 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c72e8c-cd67-4555-a76f-f78181afa5d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:54.751905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1089/2183] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:35:54.752673 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c72e8c-cd67-4555-a76f-f78181afa5d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:54.754524 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c72e8c-cd67-4555-a76f-f78181afa5d1 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:54.773071 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1095/2184] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:35:54.907515 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd39991-57d6-47a7-a1f2-10d97580b098 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:54.908165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd39991-57d6-47a7-a1f2-10d97580b098 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:54.908931 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd39991-57d6-47a7-a1f2-10d97580b098 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:54.911991 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1090/2185] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 7 13:35:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:35:54.922213 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdce790-531c-4ca9-9916-02deebb050df None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:54.922806 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdce790-531c-4ca9-9916-02deebb050df None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:54.923421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdce790-531c-4ca9-9916-02deebb050df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:54.970300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-ecdce790-531c-4ca9-9916-02deebb050df None None] MFA Rules not processed for user `97c363046db444cbb3d4346066f9ad42`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:54.972759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ecdce790-531c-4ca9-9916-02deebb050df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:55.120884 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1096/2186] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 7 13:35:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:55.169048 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd39991-57d6-47a7-a1f2-10d97580b098 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:55.175510 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5cd39991-57d6-47a7-a1f2-10d97580b098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:55.199459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd39991-57d6-47a7-a1f2-10d97580b098 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:55.203737 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f13a9-223f-47f7-8291-2781e0e7ffe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:55.204369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f13a9-223f-47f7-8291-2781e0e7ffe1 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:55.204656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f13a9-223f-47f7-8291-2781e0e7ffe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:55.204932 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f13a9-223f-47f7-8291-2781e0e7ffe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:55.218399 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f13a9-223f-47f7-8291-2781e0e7ffe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:55.219911 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f13a9-223f-47f7-8291-2781e0e7ffe1 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:55.237081 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1091/2187] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:35:55.555895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdce790-531c-4ca9-9916-02deebb050df None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:55.559356 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ecdce790-531c-4ca9-9916-02deebb050df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:55.619690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdce790-531c-4ca9-9916-02deebb050df None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:55.631000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:55.636024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:55.636024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:55.636024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2 admin admin] PATH_INFO: `/v3/projects/18372dab3b5249d5ac6ea115d41f6973` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:55.637128 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18372dab3b5249d5ac6ea115d41f6973)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:55.641829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:55.867913 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18372dab3b5249d5ac6ea115d41f6973', 'request_id': 'req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:55.890087 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18372dab3b5249d5ac6ea115d41f6973', 'request_id': 'req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:56.040383 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18372dab3b5249d5ac6ea115d41f6973', 'request_id': 'req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:56.049028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:56.073393 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2 admin admin] The token cache is being invalidate because project 18372dab3b5249d5ac6ea115d41f6973 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:35:56.074078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:56.075672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:56.079174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1097/2188] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:35:55 2022] DELETE /identity/v3/projects/18372dab3b5249d5ac6ea115d41f6973 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:56.085918 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f13a9-223f-47f7-8291-2781e0e7ffe1 service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:56.107372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f13a9-223f-47f7-8291-2781e0e7ffe1 service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:56.176116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:56.251709 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489fa91e-a667-4308-bb79-3447252893ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:56.253076 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fa91e-a667-4308-bb79-3447252893ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:56.253589 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fa91e-a667-4308-bb79-3447252893ff admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:56.254095 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fa91e-a667-4308-bb79-3447252893ff admin admin] PATH_INFO: `/v3/users/8f3b5ca828b94226a1df530725a21f0c` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:56.257136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489fa91e-a667-4308-bb79-3447252893ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f3b5ca828b94226a1df530725a21f0c)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:56.261081 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489fa91e-a667-4308-bb79-3447252893ff admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:56.692261 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-489fa91e-a667-4308-bb79-3447252893ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f3b5ca828b94226a1df530725a21f0c'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:56.722288 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-489fa91e-a667-4308-bb79-3447252893ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f3b5ca828b94226a1df530725a21f0c'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:56.724068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-489fa91e-a667-4308-bb79-3447252893ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f3b5ca828b94226a1df530725a21f0c'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:56.740932 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-489fa91e-a667-4308-bb79-3447252893ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f3b5ca828b94226a1df530725a21f0c'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:56.742704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-489fa91e-a667-4308-bb79-3447252893ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f3b5ca828b94226a1df530725a21f0c'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:56.877221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1092/2189] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:35:56 2022] DELETE /identity/v3/users/8f3b5ca828b94226a1df530725a21f0c => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:35:57.545036 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:57.547299 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:57.562752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f7f8c-5766-49fd-ad8b-4580f1ce73c2 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:57.671924 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38cda2-bc88-46c8-8af0-fffe30b5b241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:57.673090 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38cda2-bc88-46c8-8af0-fffe30b5b241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:57.673605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38cda2-bc88-46c8-8af0-fffe30b5b241 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:57.674014 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38cda2-bc88-46c8-8af0-fffe30b5b241 admin admin] PATH_INFO: `/v3/projects/9c44f21a6b424db2b29fe471713c7e65` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:57.676441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d38cda2-bc88-46c8-8af0-fffe30b5b241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c44f21a6b424db2b29fe471713c7e65)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:57.681199 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d38cda2-bc88-46c8-8af0-fffe30b5b241 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:57.827357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9d38cda2-bc88-46c8-8af0-fffe30b5b241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c44f21a6b424db2b29fe471713c7e65', 'request_id': 'req-9d38cda2-bc88-46c8-8af0-fffe30b5b241'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:57.835969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9d38cda2-bc88-46c8-8af0-fffe30b5b241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c44f21a6b424db2b29fe471713c7e65', 'request_id': 'req-9d38cda2-bc88-46c8-8af0-fffe30b5b241'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:57.931578 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9d38cda2-bc88-46c8-8af0-fffe30b5b241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c44f21a6b424db2b29fe471713c7e65', 'request_id': 'req-9d38cda2-bc88-46c8-8af0-fffe30b5b241'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:57.947388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9d38cda2-bc88-46c8-8af0-fffe30b5b241 admin admin] The token cache is being invalidate because project 9c44f21a6b424db2b29fe471713c7e65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:35:57.947728 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9d38cda2-bc88-46c8-8af0-fffe30b5b241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:57.948517 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9d38cda2-bc88-46c8-8af0-fffe30b5b241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:35:57.950275 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1098/2190] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:35:57 2022] DELETE /identity/v3/projects/9c44f21a6b424db2b29fe471713c7e65 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:57.955991 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74478237-9f39-44ab-8dab-e4dbebb7d59e None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:57.956679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74478237-9f39-44ab-8dab-e4dbebb7d59e None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:57.956679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74478237-9f39-44ab-8dab-e4dbebb7d59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:57.981431 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-74478237-9f39-44ab-8dab-e4dbebb7d59e None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:57.982721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-74478237-9f39-44ab-8dab-e4dbebb7d59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:58.060811 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1093/2191] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:58.069965 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84aeec-118c-44de-8d8d-36b5651f13eb None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:58.070239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84aeec-118c-44de-8d8d-36b5651f13eb None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:58.070610 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84aeec-118c-44de-8d8d-36b5651f13eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:58.096927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-9d84aeec-118c-44de-8d8d-36b5651f13eb None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:58.098213 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9d84aeec-118c-44de-8d8d-36b5651f13eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:58.107395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1099/2192] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:58.112604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74478237-9f39-44ab-8dab-e4dbebb7d59e None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:58.123895 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74478237-9f39-44ab-8dab-e4dbebb7d59e None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:58.129075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117fc36-78c9-4726-8444-4628cf5b7202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:58.129719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117fc36-78c9-4726-8444-4628cf5b7202 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:58.130000 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117fc36-78c9-4726-8444-4628cf5b7202 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:58.130274 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117fc36-78c9-4726-8444-4628cf5b7202 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:58.131508 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117fc36-78c9-4726-8444-4628cf5b7202 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:35:58.131940 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117fc36-78c9-4726-8444-4628cf5b7202 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:58.134093 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117fc36-78c9-4726-8444-4628cf5b7202 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:58.150240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1094/2193] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:35:58.154433 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84aeec-118c-44de-8d8d-36b5651f13eb None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:58.166040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84aeec-118c-44de-8d8d-36b5651f13eb None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:58.172720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143819e-8509-44d8-9fab-a15be39466be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:58.173386 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143819e-8509-44d8-9fab-a15be39466be admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:58.173662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143819e-8509-44d8-9fab-a15be39466be admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:58.173938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143819e-8509-44d8-9fab-a15be39466be admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:58.174636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143819e-8509-44d8-9fab-a15be39466be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:58.176875 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143819e-8509-44d8-9fab-a15be39466be admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:58.330490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1100/2194] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:35:58 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:35:58.332147 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117fc36-78c9-4726-8444-4628cf5b7202 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:58.355808 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117fc36-78c9-4726-8444-4628cf5b7202 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:58.428304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:58.472860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df73228-4c2d-4f1b-8e5d-4f33e431f75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:58.473916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df73228-4c2d-4f1b-8e5d-4f33e431f75e admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:58.474413 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df73228-4c2d-4f1b-8e5d-4f33e431f75e admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:58.475471 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df73228-4c2d-4f1b-8e5d-4f33e431f75e admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:58.479261 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df73228-4c2d-4f1b-8e5d-4f33e431f75e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:58.481805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df73228-4c2d-4f1b-8e5d-4f33e431f75e admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:58.624991 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1095/2195] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:35:58 2022] POST /identity/v3/users => generated 289 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:35:58.630749 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143819e-8509-44d8-9fab-a15be39466be admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:58.645927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143819e-8509-44d8-9fab-a15be39466be admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:58.651809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ef4cd-2932-4763-b689-e11e5d47a59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:58.652448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ef4cd-2932-4763-b689-e11e5d47a59a admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:58.652719 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ef4cd-2932-4763-b689-e11e5d47a59a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:58.652987 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ef4cd-2932-4763-b689-e11e5d47a59a admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:58.653684 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ef4cd-2932-4763-b689-e11e5d47a59a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:58.655500 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ef4cd-2932-4763-b689-e11e5d47a59a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:58.669772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1101/2196] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:35:58.674650 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df73228-4c2d-4f1b-8e5d-4f33e431f75e admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:58.688061 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df73228-4c2d-4f1b-8e5d-4f33e431f75e admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:58.695562 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2fd9e-f73d-462e-8ece-9a51d85c0baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:58.696465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2fd9e-f73d-462e-8ece-9a51d85c0baa admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:58.696935 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2fd9e-f73d-462e-8ece-9a51d85c0baa admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:58.697403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2fd9e-f73d-462e-8ece-9a51d85c0baa admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:58.698478 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2fd9e-f73d-462e-8ece-9a51d85c0baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:58.701957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2fd9e-f73d-462e-8ece-9a51d85c0baa admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:58.717590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1096/2197] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:35:58.723153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ef4cd-2932-4763-b689-e11e5d47a59a admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:58.735662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ef4cd-2932-4763-b689-e11e5d47a59a admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:58.742003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b6f51-c400-476e-bac5-d0ecce1c9d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:58.742324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b6f51-c400-476e-bac5-d0ecce1c9d94 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:58.742609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b6f51-c400-476e-bac5-d0ecce1c9d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:58.742890 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b6f51-c400-476e-bac5-d0ecce1c9d94 admin admin] PATH_INFO: `/v3/projects/9f63168911774b568e7cafb17f8a0408/users/ad0d7906c4644a798c7faa2596eea5df/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:58.774094 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b6f51-c400-476e-bac5-d0ecce1c9d94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f63168911774b568e7cafb17f8a0408, user_id=ad0d7906c4644a798c7faa2596eea5df, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:58.780190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b6f51-c400-476e-bac5-d0ecce1c9d94 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:58.902888 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1102/2198] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:35:58 2022] PUT /identity/v3/projects/9f63168911774b568e7cafb17f8a0408/users/ad0d7906c4644a798c7faa2596eea5df/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:35:58.923424 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be87a5-2d08-4caa-89fe-b4a20b3dc924 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:58.923910 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be87a5-2d08-4caa-89fe-b4a20b3dc924 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:58.924408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be87a5-2d08-4caa-89fe-b4a20b3dc924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:58.979734 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-e3be87a5-2d08-4caa-89fe-b4a20b3dc924 None None] MFA Rules not processed for user `ad0d7906c4644a798c7faa2596eea5df`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:35:58.981948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e3be87a5-2d08-4caa-89fe-b4a20b3dc924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:59.122505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1097/2199] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:35:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:35:59.139301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b6f51-c400-476e-bac5-d0ecce1c9d94 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:35:59.141859 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d25b6f51-c400-476e-bac5-d0ecce1c9d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:35:59.177281 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:35:59.233005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b6f51-c400-476e-bac5-d0ecce1c9d94 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:35:59.237273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630280ac-5321-40de-bd45-6c4fcb19bbf3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1a48805d95640489d7ce3333dd687b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:35:59.237921 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630280ac-5321-40de-bd45-6c4fcb19bbf3 service glance] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:35:59.238280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630280ac-5321-40de-bd45-6c4fcb19bbf3 service glance] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:35:59.238643 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630280ac-5321-40de-bd45-6c4fcb19bbf3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:35:59.251122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630280ac-5321-40de-bd45-6c4fcb19bbf3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:35:59.257413 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630280ac-5321-40de-bd45-6c4fcb19bbf3 service glance] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:35:59.268389 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1103/2200] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:35:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:36:09.114234 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=50,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:36:09.535113 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=18 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:36:12.021733 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb59f8b-7833-4d39-8210-4d43af21b378 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:12.024356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb59f8b-7833-4d39-8210-4d43af21b378 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:12.024356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb59f8b-7833-4d39-8210-4d43af21b378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:12.048340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-2eb59f8b-7833-4d39-8210-4d43af21b378 None None] MFA Rules not processed for user `8121ba4cf2cf44d9a5a55975ea85928c`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:36:12.049665 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2eb59f8b-7833-4d39-8210-4d43af21b378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:36:12.127995 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1098/2201] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 7 13:36:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:36:12.149326 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630280ac-5321-40de-bd45-6c4fcb19bbf3 service glance] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:36:12.151929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-630280ac-5321-40de-bd45-6c4fcb19bbf3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:36:12.268319 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630280ac-5321-40de-bd45-6c4fcb19bbf3 service glance] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:36:12.274888 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e4933-7ee3-4bed-a468-e5845e176cbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cedc7091d2f46b5be68818d26399e9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:36:12.275660 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e4933-7ee3-4bed-a468-e5845e176cbe service swift] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:12.275945 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e4933-7ee3-4bed-a468-e5845e176cbe service swift] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:12.276220 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e4933-7ee3-4bed-a468-e5845e176cbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:12.288200 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e4933-7ee3-4bed-a468-e5845e176cbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:36:12.291702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e4933-7ee3-4bed-a468-e5845e176cbe service swift] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:36:12.316013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1104/2202] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 7 13:36:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:36:22.123053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=25 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:36:22.316559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=32 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:36:26.318466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb59f8b-7833-4d39-8210-4d43af21b378 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:36:26.460637 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb59f8b-7833-4d39-8210-4d43af21b378 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:36:26.469066 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b489bf-0b3a-4ce7-b285-3e004870388d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:36:26.470260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b489bf-0b3a-4ce7-b285-3e004870388d admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:26.470741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b489bf-0b3a-4ce7-b285-3e004870388d admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:26.471182 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b489bf-0b3a-4ce7-b285-3e004870388d admin admin] PATH_INFO: `/v3/users/ad0d7906c4644a798c7faa2596eea5df` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:26.473207 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b489bf-0b3a-4ce7-b285-3e004870388d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad0d7906c4644a798c7faa2596eea5df)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:36:26.477302 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b489bf-0b3a-4ce7-b285-3e004870388d admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:36:26.732834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-12b489bf-0b3a-4ce7-b285-3e004870388d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad0d7906c4644a798c7faa2596eea5df'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:36:26.760186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-12b489bf-0b3a-4ce7-b285-3e004870388d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad0d7906c4644a798c7faa2596eea5df'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:36:26.761526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-12b489bf-0b3a-4ce7-b285-3e004870388d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad0d7906c4644a798c7faa2596eea5df'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:36:26.779868 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-12b489bf-0b3a-4ce7-b285-3e004870388d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad0d7906c4644a798c7faa2596eea5df'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:36:26.781179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-12b489bf-0b3a-4ce7-b285-3e004870388d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad0d7906c4644a798c7faa2596eea5df'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:36:26.907941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1099/2203] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:36:26 2022] DELETE /identity/v3/users/ad0d7906c4644a798c7faa2596eea5df => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:36:26.915888 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc73ba3-0432-4417-9777-6ad854388bbb None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:26.916868 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc73ba3-0432-4417-9777-6ad854388bbb None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:26.917179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc73ba3-0432-4417-9777-6ad854388bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:26.948286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-dbc73ba3-0432-4417-9777-6ad854388bbb None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:36:26.950506 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-dbc73ba3-0432-4417-9777-6ad854388bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:36:27.088024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1105/2204] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:36:27.106086 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b489bf-0b3a-4ce7-b285-3e004870388d admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:36:27.116325 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-12b489bf-0b3a-4ce7-b285-3e004870388d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:36:27.238267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b489bf-0b3a-4ce7-b285-3e004870388d admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:36:27.246351 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17133bc-ec71-4915-864e-105cfc4af162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:36:27.247550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17133bc-ec71-4915-864e-105cfc4af162 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:27.248050 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17133bc-ec71-4915-864e-105cfc4af162 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:27.248543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17133bc-ec71-4915-864e-105cfc4af162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:27.267663 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17133bc-ec71-4915-864e-105cfc4af162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:36:27.270502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17133bc-ec71-4915-864e-105cfc4af162 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:36:27.291765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1100/2205] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:36:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:36:27.943904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc73ba3-0432-4417-9777-6ad854388bbb None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:36:27.945210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-dbc73ba3-0432-4417-9777-6ad854388bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:36:27.973096 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc73ba3-0432-4417-9777-6ad854388bbb None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:36:27.981961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f019728-2f2d-4c41-9020-6583ecb9c28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:36:27.988536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f019728-2f2d-4c41-9020-6583ecb9c28c admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:27.988536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f019728-2f2d-4c41-9020-6583ecb9c28c admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:27.988536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f019728-2f2d-4c41-9020-6583ecb9c28c admin admin] PATH_INFO: `/v3/projects/9f63168911774b568e7cafb17f8a0408` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:27.988536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f019728-2f2d-4c41-9020-6583ecb9c28c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f63168911774b568e7cafb17f8a0408)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:36:27.993115 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f019728-2f2d-4c41-9020-6583ecb9c28c admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:36:28.228949 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-1f019728-2f2d-4c41-9020-6583ecb9c28c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f63168911774b568e7cafb17f8a0408', 'request_id': 'req-1f019728-2f2d-4c41-9020-6583ecb9c28c'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:36:28.238999 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-1f019728-2f2d-4c41-9020-6583ecb9c28c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f63168911774b568e7cafb17f8a0408', 'request_id': 'req-1f019728-2f2d-4c41-9020-6583ecb9c28c'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:36:28.334663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-1f019728-2f2d-4c41-9020-6583ecb9c28c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f63168911774b568e7cafb17f8a0408', 'request_id': 'req-1f019728-2f2d-4c41-9020-6583ecb9c28c'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:36:28.354729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-1f019728-2f2d-4c41-9020-6583ecb9c28c admin admin] The token cache is being invalidate because project 9f63168911774b568e7cafb17f8a0408 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:36:28.354729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-1f019728-2f2d-4c41-9020-6583ecb9c28c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:36:28.355690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-1f019728-2f2d-4c41-9020-6583ecb9c28c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:36:28.358390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1106/2206] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:36:27 2022] DELETE /identity/v3/projects/9f63168911774b568e7cafb17f8a0408 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:36:28.364706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b880a2-b83e-48e1-94ec-38b9cc9271a2 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:28.365302 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b880a2-b83e-48e1-94ec-38b9cc9271a2 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:28.365424 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b880a2-b83e-48e1-94ec-38b9cc9271a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:28.396161 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-85b880a2-b83e-48e1-94ec-38b9cc9271a2 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:36:28.398938 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-85b880a2-b83e-48e1-94ec-38b9cc9271a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:36:28.435710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:36:28.497008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1101/2207] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:36:28.505866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db502b1a-150e-423a-80d3-301207b5b2b3 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:28.506254 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db502b1a-150e-423a-80d3-301207b5b2b3 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:28.511936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db502b1a-150e-423a-80d3-301207b5b2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:28.532027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-db502b1a-150e-423a-80d3-301207b5b2b3 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:36:28.533510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-db502b1a-150e-423a-80d3-301207b5b2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:36:28.546005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1107/2208] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:36:28.550964 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b880a2-b83e-48e1-94ec-38b9cc9271a2 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:36:28.563120 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b880a2-b83e-48e1-94ec-38b9cc9271a2 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:36:28.569748 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb3915-3ec4-486f-9710-67f0b2671a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:36:28.570504 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb3915-3ec4-486f-9710-67f0b2671a7f admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:28.570831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb3915-3ec4-486f-9710-67f0b2671a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:28.571188 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb3915-3ec4-486f-9710-67f0b2671a7f admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:28.572132 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb3915-3ec4-486f-9710-67f0b2671a7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:36:28.572494 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb3915-3ec4-486f-9710-67f0b2671a7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:36:28.574572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb3915-3ec4-486f-9710-67f0b2671a7f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:36:28.592526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1102/2209] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:36:28.598559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db502b1a-150e-423a-80d3-301207b5b2b3 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:36:28.610518 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db502b1a-150e-423a-80d3-301207b5b2b3 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:36:28.616585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f1bf5-7a62-4590-b64e-d22db39560f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:36:28.617453 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1bf5-7a62-4590-b64e-d22db39560f4 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:28.617875 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1bf5-7a62-4590-b64e-d22db39560f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:28.618259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1bf5-7a62-4590-b64e-d22db39560f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:28.619093 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f1bf5-7a62-4590-b64e-d22db39560f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:36:28.621484 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f1bf5-7a62-4590-b64e-d22db39560f4 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:36:28.860531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1108/2210] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:36:28 2022] POST /identity/v3/projects => generated 371 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:36:28.867877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb3915-3ec4-486f-9710-67f0b2671a7f admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:36:28.887403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb3915-3ec4-486f-9710-67f0b2671a7f admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:36:29.037016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94273307-18cd-404f-a62c-670b46898df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:36:29.039161 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94273307-18cd-404f-a62c-670b46898df5 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:29.040114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94273307-18cd-404f-a62c-670b46898df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:29.047461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94273307-18cd-404f-a62c-670b46898df5 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:29.053229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94273307-18cd-404f-a62c-670b46898df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:36:29.056861 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94273307-18cd-404f-a62c-670b46898df5 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:36:29.188958 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1103/2211] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:36:28 2022] POST /identity/v3/users => generated 286 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:36:29.195331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f1bf5-7a62-4590-b64e-d22db39560f4 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:36:29.245287 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f1bf5-7a62-4590-b64e-d22db39560f4 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:36:29.290657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa2a5d-1144-47a1-84e3-1e52ff8e7114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:36:29.290657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa2a5d-1144-47a1-84e3-1e52ff8e7114 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:29.290657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa2a5d-1144-47a1-84e3-1e52ff8e7114 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:29.290657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa2a5d-1144-47a1-84e3-1e52ff8e7114 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:29.301286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa2a5d-1144-47a1-84e3-1e52ff8e7114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:36:29.310640 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa2a5d-1144-47a1-84e3-1e52ff8e7114 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:36:29.365064 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1109/2212] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:36:29.366805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94273307-18cd-404f-a62c-670b46898df5 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:36:29.394319 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94273307-18cd-404f-a62c-670b46898df5 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:36:29.430015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757e5bf-b129-4009-82a2-b82d55e5586c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:36:29.434142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757e5bf-b129-4009-82a2-b82d55e5586c admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:29.434142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757e5bf-b129-4009-82a2-b82d55e5586c admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:29.434142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757e5bf-b129-4009-82a2-b82d55e5586c admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:29.434142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757e5bf-b129-4009-82a2-b82d55e5586c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:36:29.436992 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757e5bf-b129-4009-82a2-b82d55e5586c admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:36:29.457265 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1104/2213] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:36:29.463830 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa2a5d-1144-47a1-84e3-1e52ff8e7114 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:36:29.488514 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa2a5d-1144-47a1-84e3-1e52ff8e7114 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:36:29.498231 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c438b-3ccb-49ea-a792-0402485006f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:36:29.499691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c438b-3ccb-49ea-a792-0402485006f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:29.500508 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c438b-3ccb-49ea-a792-0402485006f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:29.501082 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c438b-3ccb-49ea-a792-0402485006f0 admin admin] PATH_INFO: `/v3/projects/de26be40410846f38b97bf63fc307c37/users/a51f757b82dd4d35808814316e81c183/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:29.535193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c438b-3ccb-49ea-a792-0402485006f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de26be40410846f38b97bf63fc307c37, user_id=a51f757b82dd4d35808814316e81c183, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:36:29.543851 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c438b-3ccb-49ea-a792-0402485006f0 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:36:29.670491 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1110/2214] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:36:29 2022] PUT /identity/v3/projects/de26be40410846f38b97bf63fc307c37/users/a51f757b82dd4d35808814316e81c183/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:36:29.697450 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffa0c3-e277-4823-97f7-3f80e3661fd0 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:29.697946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffa0c3-e277-4823-97f7-3f80e3661fd0 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:29.698435 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffa0c3-e277-4823-97f7-3f80e3661fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:29.810138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-f3ffa0c3-e277-4823-97f7-3f80e3661fd0 None None] MFA Rules not processed for user `a51f757b82dd4d35808814316e81c183`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:36:29.812954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f3ffa0c3-e277-4823-97f7-3f80e3661fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:36:29.993437 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1105/2215] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:36:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:36:30.007608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c438b-3ccb-49ea-a792-0402485006f0 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:36:30.010368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-436c438b-3ccb-49ea-a792-0402485006f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:36:30.059230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:36:30.137498 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c438b-3ccb-49ea-a792-0402485006f0 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:36:30.144835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a14849-7706-4ff2-b07b-0ea0a0b743ec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1a48805d95640489d7ce3333dd687b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:36:30.146324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a14849-7706-4ff2-b07b-0ea0a0b743ec service glance] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:30.146730 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a14849-7706-4ff2-b07b-0ea0a0b743ec service glance] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:30.147232 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a14849-7706-4ff2-b07b-0ea0a0b743ec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:30.172891 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a14849-7706-4ff2-b07b-0ea0a0b743ec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:36:30.175916 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a14849-7706-4ff2-b07b-0ea0a0b743ec service glance] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:36:30.201347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1111/2216] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:36:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:36:39.988869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:36:40.617529 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=17 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:36:47.084547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692cd8f-745d-4b8f-acac-6a2c8c13e060 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:47.085225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692cd8f-745d-4b8f-acac-6a2c8c13e060 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:47.085504 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692cd8f-745d-4b8f-acac-6a2c8c13e060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:47.123227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-9692cd8f-745d-4b8f-acac-6a2c8c13e060 None None] MFA Rules not processed for user `8121ba4cf2cf44d9a5a55975ea85928c`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:36:47.125398 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9692cd8f-745d-4b8f-acac-6a2c8c13e060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:36:47.248624 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1106/2217] 10.222.0.62 () {58 vars in 967 bytes} [Wed Sep 7 13:36:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:36:47.288539 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a14849-7706-4ff2-b07b-0ea0a0b743ec service glance] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:36:47.291287 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c0a14849-7706-4ff2-b07b-0ea0a0b743ec service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:36:47.430951 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a14849-7706-4ff2-b07b-0ea0a0b743ec service glance] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:36:47.438143 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c44044-3c1a-4f1d-9cff-7f4c0854ad9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cedc7091d2f46b5be68818d26399e9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:36:47.439010 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c44044-3c1a-4f1d-9cff-7f4c0854ad9c service swift] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:47.439379 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c44044-3c1a-4f1d-9cff-7f4c0854ad9c service swift] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:47.439653 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c44044-3c1a-4f1d-9cff-7f4c0854ad9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:47.451917 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c44044-3c1a-4f1d-9cff-7f4c0854ad9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:36:47.454150 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c44044-3c1a-4f1d-9cff-7f4c0854ad9c service swift] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:36:47.469722 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1112/2218] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 7 13:36:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:36:47.822896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d8e07-1906-46a2-89f5-8d5d102104b0 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:47.823314 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d8e07-1906-46a2-89f5-8d5d102104b0 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:47.823794 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d8e07-1906-46a2-89f5-8d5d102104b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:47.860978 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-9e3d8e07-1906-46a2-89f5-8d5d102104b0 None None] MFA Rules not processed for user `8121ba4cf2cf44d9a5a55975ea85928c`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:36:47.863244 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9e3d8e07-1906-46a2-89f5-8d5d102104b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:36:47.878674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1107/2219] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 7 13:36:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:36:47.903457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c44044-3c1a-4f1d-9cff-7f4c0854ad9c service swift] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:36:47.926197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c44044-3c1a-4f1d-9cff-7f4c0854ad9c service swift] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:36:47.937628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b88a3-5f78-47e9-9eaa-260466da3a7a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cedc7091d2f46b5be68818d26399e9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:36:47.939149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b88a3-5f78-47e9-9eaa-260466da3a7a service swift] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:47.940082 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b88a3-5f78-47e9-9eaa-260466da3a7a service swift] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:47.940805 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b88a3-5f78-47e9-9eaa-260466da3a7a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:47.963964 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b88a3-5f78-47e9-9eaa-260466da3a7a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:36:47.968043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b88a3-5f78-47e9-9eaa-260466da3a7a service swift] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:36:48.014265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1113/2220] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 7 13:36:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:36:54.484448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92f19c-5921-46b4-849b-efd57f80fb1c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:36:54.486646 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92f19c-5921-46b4-849b-efd57f80fb1c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:36:54.487649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92f19c-5921-46b4-849b-efd57f80fb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:36:54.578046 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-9a92f19c-5921-46b4-849b-efd57f80fb1c None None] MFA Rules not processed for user `6dcde357754a4f668c30c40dc838d09f`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:36:54.589507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9a92f19c-5921-46b4-849b-efd57f80fb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:36:54.936560 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1108/2221] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:36:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:36:57.992237 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=38 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:04.907006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=53 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:16.060976 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c8719-6dab-4ea8-8362-e3278dcaa36a None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:16.061810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c8719-6dab-4ea8-8362-e3278dcaa36a None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:16.062004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c8719-6dab-4ea8-8362-e3278dcaa36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:16.109256 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b14c8719-6dab-4ea8-8362-e3278dcaa36a None None] MFA Rules not processed for user `8121ba4cf2cf44d9a5a55975ea85928c`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:37:16.112062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b14c8719-6dab-4ea8-8362-e3278dcaa36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:16.131186 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1114/2222] 10.222.0.62 () {58 vars in 967 bytes} [Wed Sep 7 13:37:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:37:16.150123 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92f19c-5921-46b4-849b-efd57f80fb1c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:16.171877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92f19c-5921-46b4-849b-efd57f80fb1c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:16.179283 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a92eda-0021-4a6c-9628-4e7a505b7c6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cedc7091d2f46b5be68818d26399e9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:16.180206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a92eda-0021-4a6c-9628-4e7a505b7c6d service swift] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:16.180582 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a92eda-0021-4a6c-9628-4e7a505b7c6d service swift] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:16.180856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a92eda-0021-4a6c-9628-4e7a505b7c6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:16.192644 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a92eda-0021-4a6c-9628-4e7a505b7c6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:16.194576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a92eda-0021-4a6c-9628-4e7a505b7c6d service swift] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:16.204464 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1109/2223] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 7 13:37:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:37:20.837846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c8719-6dab-4ea8-8362-e3278dcaa36a None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:20.839652 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b14c8719-6dab-4ea8-8362-e3278dcaa36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:20.944390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c8719-6dab-4ea8-8362-e3278dcaa36a None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:20.950449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0628cea-b161-4128-bf23-6742b999be7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:20.951381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0628cea-b161-4128-bf23-6742b999be7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:20.951777 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0628cea-b161-4128-bf23-6742b999be7a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:20.952178 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0628cea-b161-4128-bf23-6742b999be7a admin admin] PATH_INFO: `/v3/users/426ecc795ae243ffa7c7e1746e8fc4aa` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:20.953583 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0628cea-b161-4128-bf23-6742b999be7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=426ecc795ae243ffa7c7e1746e8fc4aa)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:20.955741 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0628cea-b161-4128-bf23-6742b999be7a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:21.185726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c0628cea-b161-4128-bf23-6742b999be7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '426ecc795ae243ffa7c7e1746e8fc4aa'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:21.261193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a92eda-0021-4a6c-9628-4e7a505b7c6d service swift] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:21.283016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c0628cea-b161-4128-bf23-6742b999be7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '426ecc795ae243ffa7c7e1746e8fc4aa'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:21.283016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c0628cea-b161-4128-bf23-6742b999be7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '426ecc795ae243ffa7c7e1746e8fc4aa'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:21.283016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c0628cea-b161-4128-bf23-6742b999be7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '426ecc795ae243ffa7c7e1746e8fc4aa'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:21.288766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a92eda-0021-4a6c-9628-4e7a505b7c6d service swift] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:21.292552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c0628cea-b161-4128-bf23-6742b999be7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '426ecc795ae243ffa7c7e1746e8fc4aa'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:21.305872 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c082e5-c745-4bab-9540-4eb60da83d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:21.307528 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c082e5-c745-4bab-9540-4eb60da83d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:21.308098 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c082e5-c745-4bab-9540-4eb60da83d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:21.308609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c082e5-c745-4bab-9540-4eb60da83d1d admin admin] PATH_INFO: `/v3/users/a51f757b82dd4d35808814316e81c183` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:21.311039 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c082e5-c745-4bab-9540-4eb60da83d1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a51f757b82dd4d35808814316e81c183)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:21.318145 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c082e5-c745-4bab-9540-4eb60da83d1d admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:21.325248 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1115/2224] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:37:20 2022] DELETE /identity/v3/users/426ecc795ae243ffa7c7e1746e8fc4aa => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:37:21.524392 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e6c082e5-c745-4bab-9540-4eb60da83d1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a51f757b82dd4d35808814316e81c183'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:21.552623 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e6c082e5-c745-4bab-9540-4eb60da83d1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a51f757b82dd4d35808814316e81c183'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:21.553908 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e6c082e5-c745-4bab-9540-4eb60da83d1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a51f757b82dd4d35808814316e81c183'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:21.581094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e6c082e5-c745-4bab-9540-4eb60da83d1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a51f757b82dd4d35808814316e81c183'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:21.582621 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e6c082e5-c745-4bab-9540-4eb60da83d1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a51f757b82dd4d35808814316e81c183'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:21.670967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1110/2225] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:37:21 2022] DELETE /identity/v3/users/a51f757b82dd4d35808814316e81c183 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:37:21.678253 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7f964-b1b5-4938-9b6c-adee38ecdb1c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:21.678727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7f964-b1b5-4938-9b6c-adee38ecdb1c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:21.679322 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7f964-b1b5-4938-9b6c-adee38ecdb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:21.714815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-a9f7f964-b1b5-4938-9b6c-adee38ecdb1c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:37:21.717300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a9f7f964-b1b5-4938-9b6c-adee38ecdb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:21.780661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c082e5-c745-4bab-9540-4eb60da83d1d admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:21.783279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e6c082e5-c745-4bab-9540-4eb60da83d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:21.841394 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1116/2226] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:37:21.854876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7f964-b1b5-4938-9b6c-adee38ecdb1c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:21.856663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a9f7f964-b1b5-4938-9b6c-adee38ecdb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:21.876207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:21.902812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c082e5-c745-4bab-9540-4eb60da83d1d admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:21.912510 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751312a1-0fe8-4630-85e3-ab0c8eb465a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:21.913580 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751312a1-0fe8-4630-85e3-ab0c8eb465a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:21.915187 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751312a1-0fe8-4630-85e3-ab0c8eb465a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:21.915187 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751312a1-0fe8-4630-85e3-ab0c8eb465a3 admin admin] PATH_INFO: `/v3/projects/da30824feca04ce79489eaccbb6a7001` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:21.918033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751312a1-0fe8-4630-85e3-ab0c8eb465a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da30824feca04ce79489eaccbb6a7001)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:21.924514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751312a1-0fe8-4630-85e3-ab0c8eb465a3 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:21.941621 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7f964-b1b5-4938-9b6c-adee38ecdb1c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:21.946445 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a9cb4-dddb-44e1-9a79-38487b335e7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:21.947055 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a9cb4-dddb-44e1-9a79-38487b335e7c service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:21.947342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a9cb4-dddb-44e1-9a79-38487b335e7c service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:21.947624 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a9cb4-dddb-44e1-9a79-38487b335e7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:21.960096 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a9cb4-dddb-44e1-9a79-38487b335e7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:21.961746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a9cb4-dddb-44e1-9a79-38487b335e7c service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:21.978266 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1117/2227] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:37:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:37:22.131937 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-751312a1-0fe8-4630-85e3-ab0c8eb465a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da30824feca04ce79489eaccbb6a7001', 'request_id': 'req-751312a1-0fe8-4630-85e3-ab0c8eb465a3'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:22.146280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-751312a1-0fe8-4630-85e3-ab0c8eb465a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da30824feca04ce79489eaccbb6a7001', 'request_id': 'req-751312a1-0fe8-4630-85e3-ab0c8eb465a3'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:22.240533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-751312a1-0fe8-4630-85e3-ab0c8eb465a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da30824feca04ce79489eaccbb6a7001', 'request_id': 'req-751312a1-0fe8-4630-85e3-ab0c8eb465a3'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:22.243321 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:22.268620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-751312a1-0fe8-4630-85e3-ab0c8eb465a3 admin admin] The token cache is being invalidate because project da30824feca04ce79489eaccbb6a7001 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:37:22.270259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-751312a1-0fe8-4630-85e3-ab0c8eb465a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:22.272530 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-751312a1-0fe8-4630-85e3-ab0c8eb465a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:22.274015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1111/2228] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:37:21 2022] DELETE /identity/v3/projects/da30824feca04ce79489eaccbb6a7001 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:37:22.286950 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d57b-e33a-4256-90eb-7a379e9a2d56 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:22.288200 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d57b-e33a-4256-90eb-7a379e9a2d56 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:22.288200 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98d57b-e33a-4256-90eb-7a379e9a2d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:22.328493 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-cd98d57b-e33a-4256-90eb-7a379e9a2d56 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:37:22.330290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-cd98d57b-e33a-4256-90eb-7a379e9a2d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:22.467039 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1118/2229] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:37:22.476538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac3a1a-453d-42c5-bf16-eac36829062b None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:22.476984 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac3a1a-453d-42c5-bf16-eac36829062b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:22.477481 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac3a1a-453d-42c5-bf16-eac36829062b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:22.502337 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-86ac3a1a-453d-42c5-bf16-eac36829062b None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:37:22.505571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-86ac3a1a-453d-42c5-bf16-eac36829062b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:22.515600 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1112/2230] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:37:22.522250 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98d57b-e33a-4256-90eb-7a379e9a2d56 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:22.540992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98d57b-e33a-4256-90eb-7a379e9a2d56 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:22.547493 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf95e5-7bfc-467a-b428-f76615a82135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:22.548422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf95e5-7bfc-467a-b428-f76615a82135 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:22.548791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf95e5-7bfc-467a-b428-f76615a82135 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:22.549157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf95e5-7bfc-467a-b428-f76615a82135 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:22.549758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf95e5-7bfc-467a-b428-f76615a82135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:37:22.550138 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf95e5-7bfc-467a-b428-f76615a82135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:22.553283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf95e5-7bfc-467a-b428-f76615a82135 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:22.575533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1119/2231] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:37:22.577040 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac3a1a-453d-42c5-bf16-eac36829062b None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:22.590749 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac3a1a-453d-42c5-bf16-eac36829062b None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:22.599262 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a80f7-d1fa-4e2d-93cf-979b07533326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:22.600926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a80f7-d1fa-4e2d-93cf-979b07533326 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:22.601934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a80f7-d1fa-4e2d-93cf-979b07533326 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:22.602573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a80f7-d1fa-4e2d-93cf-979b07533326 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:22.603573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a80f7-d1fa-4e2d-93cf-979b07533326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:22.606520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a80f7-d1fa-4e2d-93cf-979b07533326 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:22.762072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1113/2232] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:37:22 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:37:22.767350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf95e5-7bfc-467a-b428-f76615a82135 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:22.786665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf95e5-7bfc-467a-b428-f76615a82135 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:22.837029 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a80f7-d1fa-4e2d-93cf-979b07533326 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:22.839892 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-942a80f7-d1fa-4e2d-93cf-979b07533326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:22.890406 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b6568-626e-4acf-a27e-056a85e6f96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:22.891191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b6568-626e-4acf-a27e-056a85e6f96e admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:22.891560 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b6568-626e-4acf-a27e-056a85e6f96e admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:22.892026 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b6568-626e-4acf-a27e-056a85e6f96e admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:22.892856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b6568-626e-4acf-a27e-056a85e6f96e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:22.897734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b6568-626e-4acf-a27e-056a85e6f96e admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:22.991895 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a80f7-d1fa-4e2d-93cf-979b07533326 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:23.000187 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf357a9-8fa3-4394-a290-469776d98e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:23.001333 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf357a9-8fa3-4394-a290-469776d98e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:23.001815 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf357a9-8fa3-4394-a290-469776d98e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:23.002293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf357a9-8fa3-4394-a290-469776d98e9f admin admin] PATH_INFO: `/v3/projects/de26be40410846f38b97bf63fc307c37` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:23.005083 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf357a9-8fa3-4394-a290-469776d98e9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=de26be40410846f38b97bf63fc307c37)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:23.012033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1120/2233] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:37:22 2022] POST /identity/v3/users => generated 287 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:37:23.012341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf357a9-8fa3-4394-a290-469776d98e9f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:23.013473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b6568-626e-4acf-a27e-056a85e6f96e admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:23.028247 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b6568-626e-4acf-a27e-056a85e6f96e admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:23.036987 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd04f98-683c-49d3-ac24-942d4403be29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:23.038299 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd04f98-683c-49d3-ac24-942d4403be29 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:23.038778 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd04f98-683c-49d3-ac24-942d4403be29 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:23.039319 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd04f98-683c-49d3-ac24-942d4403be29 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:23.041264 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd04f98-683c-49d3-ac24-942d4403be29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:23.044525 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd04f98-683c-49d3-ac24-942d4403be29 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:23.067758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1121/2234] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:37:23.073283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd04f98-683c-49d3-ac24-942d4403be29 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:23.092432 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd04f98-683c-49d3-ac24-942d4403be29 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:23.101933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41434d4-6b15-4892-ab1d-fcb17019ac8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:23.103006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41434d4-6b15-4892-ab1d-fcb17019ac8f admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:23.103518 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41434d4-6b15-4892-ab1d-fcb17019ac8f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:23.103999 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41434d4-6b15-4892-ab1d-fcb17019ac8f admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:23.105269 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41434d4-6b15-4892-ab1d-fcb17019ac8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:23.108523 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41434d4-6b15-4892-ab1d-fcb17019ac8f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:23.120364 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-fcf357a9-8fa3-4394-a290-469776d98e9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de26be40410846f38b97bf63fc307c37', 'request_id': 'req-fcf357a9-8fa3-4394-a290-469776d98e9f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:23.130827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-fcf357a9-8fa3-4394-a290-469776d98e9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de26be40410846f38b97bf63fc307c37', 'request_id': 'req-fcf357a9-8fa3-4394-a290-469776d98e9f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:23.136725 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1122/2235] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:37:23.153342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41434d4-6b15-4892-ab1d-fcb17019ac8f admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:23.162795 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41434d4-6b15-4892-ab1d-fcb17019ac8f admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:23.175708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:23.247801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-fcf357a9-8fa3-4394-a290-469776d98e9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de26be40410846f38b97bf63fc307c37', 'request_id': 'req-fcf357a9-8fa3-4394-a290-469776d98e9f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:23.274525 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-fcf357a9-8fa3-4394-a290-469776d98e9f admin admin] The token cache is being invalidate because project de26be40410846f38b97bf63fc307c37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:37:23.275338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-fcf357a9-8fa3-4394-a290-469776d98e9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:23.276190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-fcf357a9-8fa3-4394-a290-469776d98e9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:23.276441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73e5e5-bdc9-4c04-83ec-c1f9d63cf563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:23.278099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73e5e5-bdc9-4c04-83ec-c1f9d63cf563 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:23.278099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73e5e5-bdc9-4c04-83ec-c1f9d63cf563 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:23.278406 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73e5e5-bdc9-4c04-83ec-c1f9d63cf563 admin admin] PATH_INFO: `/v3/projects/1ba01b2fe50e4653a1165f78f489dac9/users/b2ba2ddcaaa74ed1acc886a6beb60ac6/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:23.279695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1114/2236] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:37:22 2022] DELETE /identity/v3/projects/de26be40410846f38b97bf63fc307c37 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:37:23.288715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b0ef3-38ca-4bf6-a65b-346a80a64d54 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:23.289217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b0ef3-38ca-4bf6-a65b-346a80a64d54 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:23.289702 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b0ef3-38ca-4bf6-a65b-346a80a64d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:23.308477 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73e5e5-bdc9-4c04-83ec-c1f9d63cf563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba01b2fe50e4653a1165f78f489dac9, user_id=b2ba2ddcaaa74ed1acc886a6beb60ac6, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:23.311628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73e5e5-bdc9-4c04-83ec-c1f9d63cf563 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:23.332315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-2a2b0ef3-38ca-4bf6-a65b-346a80a64d54 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:37:23.334691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2a2b0ef3-38ca-4bf6-a65b-346a80a64d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:23.349456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1115/2237] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:37:23.358572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0104b5-88be-496a-8f16-73919b33048c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:23.359041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0104b5-88be-496a-8f16-73919b33048c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:23.359474 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0104b5-88be-496a-8f16-73919b33048c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:23.383332 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-2e0104b5-88be-496a-8f16-73919b33048c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:37:23.384822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2e0104b5-88be-496a-8f16-73919b33048c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:23.394927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1116/2238] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:37:23.400418 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0104b5-88be-496a-8f16-73919b33048c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:23.411974 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0104b5-88be-496a-8f16-73919b33048c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:23.417972 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb09a78-b26b-4797-9064-a5a24d508dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:23.418612 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb09a78-b26b-4797-9064-a5a24d508dfa admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:23.418954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb09a78-b26b-4797-9064-a5a24d508dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:23.419317 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb09a78-b26b-4797-9064-a5a24d508dfa admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:23.420201 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb09a78-b26b-4797-9064-a5a24d508dfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:37:23.420539 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb09a78-b26b-4797-9064-a5a24d508dfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:23.422174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb09a78-b26b-4797-9064-a5a24d508dfa admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:23.442421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1117/2239] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:37:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:37:23.442653 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1123/2240] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:37:23 2022] PUT /identity/v3/projects/1ba01b2fe50e4653a1165f78f489dac9/users/b2ba2ddcaaa74ed1acc886a6beb60ac6/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:37:23.444151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb09a78-b26b-4797-9064-a5a24d508dfa admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:23.459070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb09a78-b26b-4797-9064-a5a24d508dfa admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:23.460247 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515cf49-eecf-4a65-b75b-2bec1d203273 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:23.460760 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515cf49-eecf-4a65-b75b-2bec1d203273 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:23.461062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515cf49-eecf-4a65-b75b-2bec1d203273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:23.504131 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:23.504809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-0515cf49-eecf-4a65-b75b-2bec1d203273 None None] MFA Rules not processed for user `b2ba2ddcaaa74ed1acc886a6beb60ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:37:23.506495 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0515cf49-eecf-4a65-b75b-2bec1d203273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:23.580997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cf1e3-f12f-49e8-aa6b-52a35fa19f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:23.582731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cf1e3-f12f-49e8-aa6b-52a35fa19f4b admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:23.583213 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cf1e3-f12f-49e8-aa6b-52a35fa19f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:23.583572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cf1e3-f12f-49e8-aa6b-52a35fa19f4b admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:23.584439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9cf1e3-f12f-49e8-aa6b-52a35fa19f4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:23.586747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9cf1e3-f12f-49e8-aa6b-52a35fa19f4b admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:23.638246 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1124/2241] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:37:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:37:23.651742 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515cf49-eecf-4a65-b75b-2bec1d203273 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:23.653210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0515cf49-eecf-4a65-b75b-2bec1d203273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:23.729398 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1118/2242] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:37:23 2022] POST /identity/v3/projects => generated 373 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:37:23.734437 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cf1e3-f12f-49e8-aa6b-52a35fa19f4b admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:23.741096 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515cf49-eecf-4a65-b75b-2bec1d203273 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:23.838631 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:23.843802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cf1e3-f12f-49e8-aa6b-52a35fa19f4b admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:23.851198 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86558f-4eb2-47ef-bf62-ac048bb1915e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:23.856300 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86558f-4eb2-47ef-bf62-ac048bb1915e admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:23.856300 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86558f-4eb2-47ef-bf62-ac048bb1915e admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:23.856300 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86558f-4eb2-47ef-bf62-ac048bb1915e admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:23.856300 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86558f-4eb2-47ef-bf62-ac048bb1915e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:23.856300 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86558f-4eb2-47ef-bf62-ac048bb1915e admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:23.863468 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ef5f4-aa11-4b89-af64-6ab14954a737 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:23.865652 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ef5f4-aa11-4b89-af64-6ab14954a737 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:23.866469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ef5f4-aa11-4b89-af64-6ab14954a737 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:23.867285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ef5f4-aa11-4b89-af64-6ab14954a737 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:24.011453 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ef5f4-aa11-4b89-af64-6ab14954a737 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:24.011453 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ef5f4-aa11-4b89-af64-6ab14954a737 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:24.012758 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1119/2243] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:37:23 2022] POST /identity/v3/users => generated 287 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:37:24.018426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86558f-4eb2-47ef-bf62-ac048bb1915e admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:24.051036 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1125/2244] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:37:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:37:24.080810 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86558f-4eb2-47ef-bf62-ac048bb1915e admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:24.113824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8021d-088d-4c5b-8160-ee420b0dd2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:24.113824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8021d-088d-4c5b-8160-ee420b0dd2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:24.113824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8021d-088d-4c5b-8160-ee420b0dd2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:24.113824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8021d-088d-4c5b-8160-ee420b0dd2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:24.114415 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8021d-088d-4c5b-8160-ee420b0dd2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:24.114415 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8021d-088d-4c5b-8160-ee420b0dd2f1 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:24.114673 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ef5f4-aa11-4b89-af64-6ab14954a737 service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:24.116496 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9b1ef5f4-aa11-4b89-af64-6ab14954a737 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:24.138651 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ef5f4-aa11-4b89-af64-6ab14954a737 service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:24.145018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d7239-5b92-4e72-90ac-3d0a0fa40ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:24.147245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1120/2245] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:37:24.147670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d7239-5b92-4e72-90ac-3d0a0fa40ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:24.147670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d7239-5b92-4e72-90ac-3d0a0fa40ded admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:24.147670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d7239-5b92-4e72-90ac-3d0a0fa40ded admin admin] PATH_INFO: `/v3/users/b2ba2ddcaaa74ed1acc886a6beb60ac6` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:24.151673 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d7239-5b92-4e72-90ac-3d0a0fa40ded admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2ba2ddcaaa74ed1acc886a6beb60ac6)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:24.161559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8021d-088d-4c5b-8160-ee420b0dd2f1 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:24.161798 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d7239-5b92-4e72-90ac-3d0a0fa40ded admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:24.168958 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8021d-088d-4c5b-8160-ee420b0dd2f1 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:24.176967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50c5a4-ee2f-425f-b758-59b76dacf799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:24.178857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50c5a4-ee2f-425f-b758-59b76dacf799 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:24.178857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50c5a4-ee2f-425f-b758-59b76dacf799 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:24.178857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50c5a4-ee2f-425f-b758-59b76dacf799 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:24.179411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50c5a4-ee2f-425f-b758-59b76dacf799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:24.180705 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50c5a4-ee2f-425f-b758-59b76dacf799 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:24.198241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1121/2246] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:37:24.202888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50c5a4-ee2f-425f-b758-59b76dacf799 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:24.216746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50c5a4-ee2f-425f-b758-59b76dacf799 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:24.222049 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02c527-dadf-4ba8-ba09-a400514362e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:24.223195 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02c527-dadf-4ba8-ba09-a400514362e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:24.223521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02c527-dadf-4ba8-ba09-a400514362e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:24.223799 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02c527-dadf-4ba8-ba09-a400514362e1 admin admin] PATH_INFO: `/v3/projects/87e706f3406f489bb619ac3adb7fdf97/users/37bf40a27a4c42d48a804701dc599671/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:24.251213 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02c527-dadf-4ba8-ba09-a400514362e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e706f3406f489bb619ac3adb7fdf97, user_id=37bf40a27a4c42d48a804701dc599671, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:24.255432 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02c527-dadf-4ba8-ba09-a400514362e1 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:24.355629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1122/2247] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:37:24 2022] PUT /identity/v3/projects/87e706f3406f489bb619ac3adb7fdf97/users/37bf40a27a4c42d48a804701dc599671/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:37:24.369201 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cf10f-2de7-42e3-99c2-0c0cec5dbf6a None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:24.369536 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cf10f-2de7-42e3-99c2-0c0cec5dbf6a None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:24.369888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cf10f-2de7-42e3-99c2-0c0cec5dbf6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:24.404428 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-df0cf10f-2de7-42e3-99c2-0c0cec5dbf6a None None] MFA Rules not processed for user `37bf40a27a4c42d48a804701dc599671`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:37:24.405680 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-df0cf10f-2de7-42e3-99c2-0c0cec5dbf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:24.450851 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-787d7239-5b92-4e72-90ac-3d0a0fa40ded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2ba2ddcaaa74ed1acc886a6beb60ac6'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:24.455890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:24.482665 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1123/2248] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:37:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:37:24.494455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cf10f-2de7-42e3-99c2-0c0cec5dbf6a None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:24.496005 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-df0cf10f-2de7-42e3-99c2-0c0cec5dbf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:24.540634 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-787d7239-5b92-4e72-90ac-3d0a0fa40ded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2ba2ddcaaa74ed1acc886a6beb60ac6'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:24.541466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-787d7239-5b92-4e72-90ac-3d0a0fa40ded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2ba2ddcaaa74ed1acc886a6beb60ac6'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:24.542198 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-787d7239-5b92-4e72-90ac-3d0a0fa40ded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2ba2ddcaaa74ed1acc886a6beb60ac6'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:24.552406 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-787d7239-5b92-4e72-90ac-3d0a0fa40ded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2ba2ddcaaa74ed1acc886a6beb60ac6'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:24.582497 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1126/2249] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:37:24 2022] DELETE /identity/v3/users/b2ba2ddcaaa74ed1acc886a6beb60ac6 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:37:24.588207 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390d42c-958a-45a2-9baf-98db0156bd5a None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:24.588606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390d42c-958a-45a2-9baf-98db0156bd5a None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:24.588977 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390d42c-958a-45a2-9baf-98db0156bd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:24.590638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cf10f-2de7-42e3-99c2-0c0cec5dbf6a None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:24.618302 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-a390d42c-958a-45a2-9baf-98db0156bd5a None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:37:24.620425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a390d42c-958a-45a2-9baf-98db0156bd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:24.678137 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfeb65-0bf3-4c9b-89ba-e017e5f88d26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:24.679291 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfeb65-0bf3-4c9b-89ba-e017e5f88d26 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:24.679656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfeb65-0bf3-4c9b-89ba-e017e5f88d26 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:24.681257 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfeb65-0bf3-4c9b-89ba-e017e5f88d26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:24.683436 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-1acfeb65-0bf3-4c9b-89ba-e017e5f88d26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:24.729759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1127/2250] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:37:24.740677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390d42c-958a-45a2-9baf-98db0156bd5a None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:24.742443 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a390d42c-958a-45a2-9baf-98db0156bd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:24.755437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:24.765276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfeb65-0bf3-4c9b-89ba-e017e5f88d26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:24.766772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfeb65-0bf3-4c9b-89ba-e017e5f88d26 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:24.779701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1124/2251] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:37:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:37:24.815365 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390d42c-958a-45a2-9baf-98db0156bd5a None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:24.821468 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c2ca4-6688-4564-93f2-1529e52eeb2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:24.822418 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c2ca4-6688-4564-93f2-1529e52eeb2b service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:24.822418 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c2ca4-6688-4564-93f2-1529e52eeb2b service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:24.822906 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c2ca4-6688-4564-93f2-1529e52eeb2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:24.837151 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c2ca4-6688-4564-93f2-1529e52eeb2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:24.844443 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c2ca4-6688-4564-93f2-1529e52eeb2b service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:24.852296 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1128/2252] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:37:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:37:25.586030 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfeb65-0bf3-4c9b-89ba-e017e5f88d26 service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:25.588668 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-1acfeb65-0bf3-4c9b-89ba-e017e5f88d26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:25.605741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfeb65-0bf3-4c9b-89ba-e017e5f88d26 service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:25.610688 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:25.616041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:25.616041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:25.616041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba admin admin] PATH_INFO: `/v3/projects/1ba01b2fe50e4653a1165f78f489dac9` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:25.616041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ba01b2fe50e4653a1165f78f489dac9)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:25.616877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:25.800001 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:25.802259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ba01b2fe50e4653a1165f78f489dac9', 'request_id': 'req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:25.816484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ba01b2fe50e4653a1165f78f489dac9', 'request_id': 'req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:25.912918 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ba01b2fe50e4653a1165f78f489dac9', 'request_id': 'req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:25.928445 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba admin admin] The token cache is being invalidate because project 1ba01b2fe50e4653a1165f78f489dac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:37:25.928743 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:25.929433 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:25.930968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1125/2253] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:37:25 2022] DELETE /identity/v3/projects/1ba01b2fe50e4653a1165f78f489dac9 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:37:26.436483 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c2ca4-6688-4564-93f2-1529e52eeb2b service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:26.438394 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-020c2ca4-6688-4564-93f2-1529e52eeb2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:26.454798 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c2ca4-6688-4564-93f2-1529e52eeb2b service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:26.541393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:26.542935 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:26.549849 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e8916-95ae-42f3-9861-7d6c2198805d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:26.551016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e8916-95ae-42f3-9861-7d6c2198805d service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:26.551442 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e8916-95ae-42f3-9861-7d6c2198805d service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:26.551909 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e8916-95ae-42f3-9861-7d6c2198805d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:26.554572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-968e8916-95ae-42f3-9861-7d6c2198805d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:26.647324 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd6d49-bcfd-45ab-a3f0-a1466e85caba admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:26.653381 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce282eb-4956-450b-93fc-42a851bbfce8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:26.654411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce282eb-4956-450b-93fc-42a851bbfce8 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:26.654862 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce282eb-4956-450b-93fc-42a851bbfce8 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:26.655359 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce282eb-4956-450b-93fc-42a851bbfce8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:26.658163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2ce282eb-4956-450b-93fc-42a851bbfce8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:26.665349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e8916-95ae-42f3-9861-7d6c2198805d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:26.668145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e8916-95ae-42f3-9861-7d6c2198805d service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:26.681831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce282eb-4956-450b-93fc-42a851bbfce8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:26.683357 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce282eb-4956-450b-93fc-42a851bbfce8 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:26.685410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1129/2254] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:37:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:37:26.698065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1126/2255] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:37:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:37:36.679337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=77 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:36.691778 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=33,keystone:INSERT=1,keystone:DELETE=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:40.098172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e8916-95ae-42f3-9861-7d6c2198805d service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:40.102227 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-968e8916-95ae-42f3-9861-7d6c2198805d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:40.227494 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e8916-95ae-42f3-9861-7d6c2198805d service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:40.240238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9679fc-e08c-4934-aec7-56ed2800d0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:40.241255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9679fc-e08c-4934-aec7-56ed2800d0c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:40.241631 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9679fc-e08c-4934-aec7-56ed2800d0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:40.242005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9679fc-e08c-4934-aec7-56ed2800d0c8 admin admin] PATH_INFO: `/v3/users/37bf40a27a4c42d48a804701dc599671` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:40.244396 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9679fc-e08c-4934-aec7-56ed2800d0c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37bf40a27a4c42d48a804701dc599671)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:40.248562 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9679fc-e08c-4934-aec7-56ed2800d0c8 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:40.491252 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-1a9679fc-e08c-4934-aec7-56ed2800d0c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37bf40a27a4c42d48a804701dc599671'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:40.615194 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-1a9679fc-e08c-4934-aec7-56ed2800d0c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37bf40a27a4c42d48a804701dc599671'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:40.616339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-1a9679fc-e08c-4934-aec7-56ed2800d0c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37bf40a27a4c42d48a804701dc599671'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:40.617443 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-1a9679fc-e08c-4934-aec7-56ed2800d0c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37bf40a27a4c42d48a804701dc599671'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:40.635399 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-1a9679fc-e08c-4934-aec7-56ed2800d0c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37bf40a27a4c42d48a804701dc599671'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:40.680341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1130/2256] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:37:40 2022] DELETE /identity/v3/users/37bf40a27a4c42d48a804701dc599671 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:37:40.687350 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8cd0c-1a90-4489-a643-3ede46023118 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:40.688072 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8cd0c-1a90-4489-a643-3ede46023118 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:40.688374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8cd0c-1a90-4489-a643-3ede46023118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:40.724053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-eff8cd0c-1a90-4489-a643-3ede46023118 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:37:40.725248 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-eff8cd0c-1a90-4489-a643-3ede46023118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:40.853796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1127/2257] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:37:40.870797 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9679fc-e08c-4934-aec7-56ed2800d0c8 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:40.873732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1a9679fc-e08c-4934-aec7-56ed2800d0c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:41.001504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9679fc-e08c-4934-aec7-56ed2800d0c8 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:41.008561 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7e656-7f8b-4404-b412-e23a2c0a346d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:41.009315 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7e656-7f8b-4404-b412-e23a2c0a346d service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:41.009593 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7e656-7f8b-4404-b412-e23a2c0a346d service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:41.009862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7e656-7f8b-4404-b412-e23a2c0a346d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:41.023585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7e656-7f8b-4404-b412-e23a2c0a346d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:41.025275 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7e656-7f8b-4404-b412-e23a2c0a346d service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:41.038011 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1131/2258] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:37:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:37:42.107643 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8cd0c-1a90-4489-a643-3ede46023118 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:42.112156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-eff8cd0c-1a90-4489-a643-3ede46023118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:42.131469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8cd0c-1a90-4489-a643-3ede46023118 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:42.140721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b19c14-39f5-4698-9649-e586a55f98af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:42.141396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b19c14-39f5-4698-9649-e586a55f98af admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:42.141670 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b19c14-39f5-4698-9649-e586a55f98af admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:42.141946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b19c14-39f5-4698-9649-e586a55f98af admin admin] PATH_INFO: `/v3/projects/87e706f3406f489bb619ac3adb7fdf97` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:42.143930 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b19c14-39f5-4698-9649-e586a55f98af admin admin] RBAC: Authorizing `identity:delete_project(project_id=87e706f3406f489bb619ac3adb7fdf97)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:42.147145 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b19c14-39f5-4698-9649-e586a55f98af admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:42.325294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f4b19c14-39f5-4698-9649-e586a55f98af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87e706f3406f489bb619ac3adb7fdf97', 'request_id': 'req-f4b19c14-39f5-4698-9649-e586a55f98af'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:42.343069 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f4b19c14-39f5-4698-9649-e586a55f98af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87e706f3406f489bb619ac3adb7fdf97', 'request_id': 'req-f4b19c14-39f5-4698-9649-e586a55f98af'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:42.443660 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f4b19c14-39f5-4698-9649-e586a55f98af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87e706f3406f489bb619ac3adb7fdf97', 'request_id': 'req-f4b19c14-39f5-4698-9649-e586a55f98af'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:42.465263 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f4b19c14-39f5-4698-9649-e586a55f98af admin admin] The token cache is being invalidate because project 87e706f3406f489bb619ac3adb7fdf97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:37:42.465647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f4b19c14-39f5-4698-9649-e586a55f98af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:42.466516 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f4b19c14-39f5-4698-9649-e586a55f98af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:37:42.469173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1128/2259] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:37:42 2022] DELETE /identity/v3/projects/87e706f3406f489bb619ac3adb7fdf97 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:37:42.475041 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d02b0e-13f9-4e00-aad0-3eaefbc5ce99 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:42.475902 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d02b0e-13f9-4e00-aad0-3eaefbc5ce99 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:42.475902 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d02b0e-13f9-4e00-aad0-3eaefbc5ce99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:42.538105 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-25d02b0e-13f9-4e00-aad0-3eaefbc5ce99 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:37:42.541814 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-25d02b0e-13f9-4e00-aad0-3eaefbc5ce99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:42.581887 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:42.667626 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127e8f9-60e8-4ecc-8cb0-1adcbffd4c9b None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:42.668120 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127e8f9-60e8-4ecc-8cb0-1adcbffd4c9b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:42.668605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127e8f9-60e8-4ecc-8cb0-1adcbffd4c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:42.681718 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1132/2260] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:37:42.691794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09350cbb-332b-4f55-a565-5b2019c09283 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:42.692545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09350cbb-332b-4f55-a565-5b2019c09283 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:42.692702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09350cbb-332b-4f55-a565-5b2019c09283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:42.713681 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-6127e8f9-60e8-4ecc-8cb0-1adcbffd4c9b None None] MFA Rules not processed for user `c5e931aa22344990a0f032159ebc20ad`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:37:42.713681 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6127e8f9-60e8-4ecc-8cb0-1adcbffd4c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:42.729459 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-09350cbb-332b-4f55-a565-5b2019c09283 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:37:42.731068 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-09350cbb-332b-4f55-a565-5b2019c09283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:42.759586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1133/2261] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:37:42.759586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09350cbb-332b-4f55-a565-5b2019c09283 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:42.769391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09350cbb-332b-4f55-a565-5b2019c09283 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:42.776631 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f28186-0685-413a-a382-a67587f990f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:42.777797 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f28186-0685-413a-a382-a67587f990f3 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:42.778325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f28186-0685-413a-a382-a67587f990f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:42.778827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f28186-0685-413a-a382-a67587f990f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:42.780535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f28186-0685-413a-a382-a67587f990f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:37:42.781073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f28186-0685-413a-a382-a67587f990f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:42.784289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f28186-0685-413a-a382-a67587f990f3 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:42.813982 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1134/2262] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:37:42.826663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f28186-0685-413a-a382-a67587f990f3 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:42.850138 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f28186-0685-413a-a382-a67587f990f3 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:42.887216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f53ee-a78b-4a33-b94a-89b3980ae521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:42.888627 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f53ee-a78b-4a33-b94a-89b3980ae521 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:42.889304 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f53ee-a78b-4a33-b94a-89b3980ae521 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:42.889988 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f53ee-a78b-4a33-b94a-89b3980ae521 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:42.892563 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f53ee-a78b-4a33-b94a-89b3980ae521 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:42.898828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f53ee-a78b-4a33-b94a-89b3980ae521 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:42.948068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1129/2263] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:37:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:37:43.070611 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1135/2264] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:37:42 2022] POST /identity/v3/projects => generated 379 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:37:43.077827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127e8f9-60e8-4ecc-8cb0-1adcbffd4c9b None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:43.107391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127e8f9-60e8-4ecc-8cb0-1adcbffd4c9b None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:43.158360 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:43.239182 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f099c9-353a-4355-aeb7-9e5baf55abef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:43.240819 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f099c9-353a-4355-aeb7-9e5baf55abef admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:43.241431 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f099c9-353a-4355-aeb7-9e5baf55abef admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:43.242065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f099c9-353a-4355-aeb7-9e5baf55abef admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:43.243671 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f099c9-353a-4355-aeb7-9e5baf55abef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:43.249016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f099c9-353a-4355-aeb7-9e5baf55abef admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:43.412474 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1130/2265] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:37:43 2022] POST /identity/v3/users => generated 290 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:37:43.412755 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f53ee-a78b-4a33-b94a-89b3980ae521 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:43.456591 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f53ee-a78b-4a33-b94a-89b3980ae521 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:43.468668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc3b1c-8bda-45f2-ad1e-8e7a98fc1a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:43.469844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc3b1c-8bda-45f2-ad1e-8e7a98fc1a5a admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:43.473259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc3b1c-8bda-45f2-ad1e-8e7a98fc1a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:43.473259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc3b1c-8bda-45f2-ad1e-8e7a98fc1a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:43.473259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc3b1c-8bda-45f2-ad1e-8e7a98fc1a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:43.476064 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc3b1c-8bda-45f2-ad1e-8e7a98fc1a5a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:43.522179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1136/2266] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:37:43.523720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f099c9-353a-4355-aeb7-9e5baf55abef admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:43.548231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f099c9-353a-4355-aeb7-9e5baf55abef admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:43.557194 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043d33d-a769-4422-8f23-65ae5d780850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:43.558283 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043d33d-a769-4422-8f23-65ae5d780850 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:43.558774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043d33d-a769-4422-8f23-65ae5d780850 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:43.559416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043d33d-a769-4422-8f23-65ae5d780850 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:43.561158 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043d33d-a769-4422-8f23-65ae5d780850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:43.564707 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043d33d-a769-4422-8f23-65ae5d780850 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:43.586172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1131/2267] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:37:43.593179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc3b1c-8bda-45f2-ad1e-8e7a98fc1a5a admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:43.612939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc3b1c-8bda-45f2-ad1e-8e7a98fc1a5a admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:43.622039 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d524c61-6513-461b-adf2-f62731a3a065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:43.623251 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d524c61-6513-461b-adf2-f62731a3a065 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:43.623783 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d524c61-6513-461b-adf2-f62731a3a065 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:43.624347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d524c61-6513-461b-adf2-f62731a3a065 admin admin] PATH_INFO: `/v3/projects/b4d5ce9b4fe34fc094bdacc3ca994537/users/9dae215cd92f46d18a3c1b7389c3af4b/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:43.661626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d524c61-6513-461b-adf2-f62731a3a065 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4d5ce9b4fe34fc094bdacc3ca994537, user_id=9dae215cd92f46d18a3c1b7389c3af4b, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:43.668728 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d524c61-6513-461b-adf2-f62731a3a065 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:43.830491 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1137/2268] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:37:43 2022] PUT /identity/v3/projects/b4d5ce9b4fe34fc094bdacc3ca994537/users/9dae215cd92f46d18a3c1b7389c3af4b/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:37:43.840075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe84b66-57ad-41ef-8dc0-96413c37ffa7 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:43.840595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe84b66-57ad-41ef-8dc0-96413c37ffa7 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:43.842551 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe84b66-57ad-41ef-8dc0-96413c37ffa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:43.890353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-ebe84b66-57ad-41ef-8dc0-96413c37ffa7 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:37:43.892457 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ebe84b66-57ad-41ef-8dc0-96413c37ffa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:44.039113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1132/2269] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:37:44.049871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d524c61-6513-461b-adf2-f62731a3a065 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:44.052833 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-3d524c61-6513-461b-adf2-f62731a3a065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:44.156975 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d524c61-6513-461b-adf2-f62731a3a065 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:44.162737 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29fe4f-e1ad-4c56-b72e-e21a14c49993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:44.163487 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29fe4f-e1ad-4c56-b72e-e21a14c49993 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:44.171313 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29fe4f-e1ad-4c56-b72e-e21a14c49993 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:44.171313 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29fe4f-e1ad-4c56-b72e-e21a14c49993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:44.183182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29fe4f-e1ad-4c56-b72e-e21a14c49993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:44.186559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29fe4f-e1ad-4c56-b72e-e21a14c49993 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:44.210569 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1138/2270] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:37:46.134118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe84b66-57ad-41ef-8dc0-96413c37ffa7 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:46.135759 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ebe84b66-57ad-41ef-8dc0-96413c37ffa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:46.246288 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe84b66-57ad-41ef-8dc0-96413c37ffa7 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:46.258127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589130f9-6fab-4cc9-bcc8-fbfaed7d81f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:46.259655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589130f9-6fab-4cc9-bcc8-fbfaed7d81f9 service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:46.260943 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589130f9-6fab-4cc9-bcc8-fbfaed7d81f9 service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:46.261494 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589130f9-6fab-4cc9-bcc8-fbfaed7d81f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:46.266871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-589130f9-6fab-4cc9-bcc8-fbfaed7d81f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:46.368627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:46.388294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589130f9-6fab-4cc9-bcc8-fbfaed7d81f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:46.390382 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589130f9-6fab-4cc9-bcc8-fbfaed7d81f9 service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:46.402606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1133/2271] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:37:46 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:37:46.676749 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9d1cf-0996-4148-b4fa-2bff84cab7bb None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:46.677458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9d1cf-0996-4148-b4fa-2bff84cab7bb None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:46.677458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9d1cf-0996-4148-b4fa-2bff84cab7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:46.707675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-0ae9d1cf-0996-4148-b4fa-2bff84cab7bb None None] MFA Rules not processed for user `a145e63a44db43acafc92bb2a238f463`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:37:46.709107 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0ae9d1cf-0996-4148-b4fa-2bff84cab7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:46.717852 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1139/2272] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:37:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:37:54.722567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d1a6a-7786-4dd4-a826-08ef1ef75cc6 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:54.722567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d1a6a-7786-4dd4-a826-08ef1ef75cc6 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:54.722567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d1a6a-7786-4dd4-a826-08ef1ef75cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:54.772930 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-b45d1a6a-7786-4dd4-a826-08ef1ef75cc6 None None] MFA Rules not processed for user `32401931d75649018f6b0e55144bb5f2`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:37:54.774460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b45d1a6a-7786-4dd4-a826-08ef1ef75cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:54.935181 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1134/2273] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:37:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:37:55.864924 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae70b4b-fe4b-4bc3-baa9-44b069ff727b None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:55.865860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae70b4b-fe4b-4bc3-baa9-44b069ff727b None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:55.866075 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae70b4b-fe4b-4bc3-baa9-44b069ff727b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:55.936613 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-7ae70b4b-fe4b-4bc3-baa9-44b069ff727b None None] MFA Rules not processed for user `9dae215cd92f46d18a3c1b7389c3af4b`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:37:55.938460 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7ae70b4b-fe4b-4bc3-baa9-44b069ff727b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:56.015871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:56.090945 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1140/2274] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:37:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:37:56.099543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d1a6a-7786-4dd4-a826-08ef1ef75cc6 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:56.120007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d1a6a-7786-4dd4-a826-08ef1ef75cc6 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:56.132942 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ebc7d-0f54-4853-b48e-476d4648efc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:56.134795 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ebc7d-0f54-4853-b48e-476d4648efc8 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:56.135436 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ebc7d-0f54-4853-b48e-476d4648efc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:56.136430 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ebc7d-0f54-4853-b48e-476d4648efc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:56.181111 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ebc7d-0f54-4853-b48e-476d4648efc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:56.181111 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ebc7d-0f54-4853-b48e-476d4648efc8 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:56.318216 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1135/2275] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:37:56 2022] POST /identity/v3/projects => generated 379 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:37:56.322785 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae70b4b-fe4b-4bc3-baa9-44b069ff727b None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:56.338009 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae70b4b-fe4b-4bc3-baa9-44b069ff727b None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:56.450143 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc9fb9-e499-4b93-ac40-5c087458ce63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:56.452060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc9fb9-e499-4b93-ac40-5c087458ce63 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:56.453336 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc9fb9-e499-4b93-ac40-5c087458ce63 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:56.453336 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc9fb9-e499-4b93-ac40-5c087458ce63 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:56.455065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc9fb9-e499-4b93-ac40-5c087458ce63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:56.459666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc9fb9-e499-4b93-ac40-5c087458ce63 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:56.641862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1141/2276] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:37:56 2022] POST /identity/v3/users => generated 296 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:37:56.647044 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ebc7d-0f54-4853-b48e-476d4648efc8 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:56.684079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ebc7d-0f54-4853-b48e-476d4648efc8 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:56.710287 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc8bc8-475b-43d2-846f-680be6762586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:56.711188 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc8bc8-475b-43d2-846f-680be6762586 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:56.711533 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc8bc8-475b-43d2-846f-680be6762586 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:56.712890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc8bc8-475b-43d2-846f-680be6762586 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:56.713791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc8bc8-475b-43d2-846f-680be6762586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:56.717865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc8bc8-475b-43d2-846f-680be6762586 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:56.740471 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1136/2277] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:37:56.742230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc9fb9-e499-4b93-ac40-5c087458ce63 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:56.766316 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc9fb9-e499-4b93-ac40-5c087458ce63 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:56.779311 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b72d3-5cb5-4d32-820b-3d1f9f3b4485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:56.781483 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b72d3-5cb5-4d32-820b-3d1f9f3b4485 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:56.784393 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b72d3-5cb5-4d32-820b-3d1f9f3b4485 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:56.786860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b72d3-5cb5-4d32-820b-3d1f9f3b4485 admin admin] PATH_INFO: `/v3/projects/605c6eb7c686455998808c1c39edb8cb/users/1b7d4a431f1c4feca7d33fcdb8ec1b7b/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:56.833547 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747b72d3-5cb5-4d32-820b-3d1f9f3b4485 admin admin] RBAC: Authorizing `identity:create_grant(project_id=605c6eb7c686455998808c1c39edb8cb, user_id=1b7d4a431f1c4feca7d33fcdb8ec1b7b, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:56.839988 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747b72d3-5cb5-4d32-820b-3d1f9f3b4485 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:56.996563 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc8bc8-475b-43d2-846f-680be6762586 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:57.007280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a7dc8bc8-475b-43d2-846f-680be6762586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:57.037718 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1142/2278] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:37:56 2022] PUT /identity/v3/projects/605c6eb7c686455998808c1c39edb8cb/users/1b7d4a431f1c4feca7d33fcdb8ec1b7b/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:37:57.044909 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b72d3-5cb5-4d32-820b-3d1f9f3b4485 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:57.200162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc8bc8-475b-43d2-846f-680be6762586 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:57.251981 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b72d3-5cb5-4d32-820b-3d1f9f3b4485 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:57.270395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ea3c1-31d0-46de-9eed-17bb070997ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:57.271570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ea3c1-31d0-46de-9eed-17bb070997ac admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:57.272440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ea3c1-31d0-46de-9eed-17bb070997ac admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:57.273030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ea3c1-31d0-46de-9eed-17bb070997ac admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:57.274900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ea3c1-31d0-46de-9eed-17bb070997ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:57.279267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ea3c1-31d0-46de-9eed-17bb070997ac admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:57.293910 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:37:57.308877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7af9a8-faf3-432f-a9bd-2f9c1d4632ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bee90bf951f04c2e8e19ceeacb8cfcb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:57.315010 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1143/2279] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:37:57.315311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7af9a8-faf3-432f-a9bd-2f9c1d4632ab service placement] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:57.315311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7af9a8-faf3-432f-a9bd-2f9c1d4632ab service placement] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:57.315311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7af9a8-faf3-432f-a9bd-2f9c1d4632ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:57.317942 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ea3c1-31d0-46de-9eed-17bb070997ac admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:37:57.322278 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9b7af9a8-faf3-432f-a9bd-2f9c1d4632ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:37:57.341153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ea3c1-31d0-46de-9eed-17bb070997ac admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:37:57.346259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7af9a8-faf3-432f-a9bd-2f9c1d4632ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:57.349971 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7af9a8-faf3-432f-a9bd-2f9c1d4632ab service placement] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:57.351093 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15a2c0-4d14-47a2-a332-944aa7cfdef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:37:57.352178 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15a2c0-4d14-47a2-a332-944aa7cfdef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:37:57.353458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15a2c0-4d14-47a2-a332-944aa7cfdef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:37:57.353458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15a2c0-4d14-47a2-a332-944aa7cfdef1 admin admin] PATH_INFO: `/v3/projects/605c6eb7c686455998808c1c39edb8cb/users/1b7d4a431f1c4feca7d33fcdb8ec1b7b/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:37:57.356711 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15a2c0-4d14-47a2-a332-944aa7cfdef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=605c6eb7c686455998808c1c39edb8cb, user_id=1b7d4a431f1c4feca7d33fcdb8ec1b7b, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:37:57.361316 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15a2c0-4d14-47a2-a332-944aa7cfdef1 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:37:57.383331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1137/2281] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:37:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:37:57.383631 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1144/2281] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:37:57 2022] PUT /identity/v3/projects/605c6eb7c686455998808c1c39edb8cb/users/1b7d4a431f1c4feca7d33fcdb8ec1b7b/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:38:00.084738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15a2c0-4d14-47a2-a332-944aa7cfdef1 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:38:00.125404 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:38:00.240982 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15a2c0-4d14-47a2-a332-944aa7cfdef1 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:38:00.251068 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14094fb3-9246-4348-a7bc-30892b3b19e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:38:00.253045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14094fb3-9246-4348-a7bc-30892b3b19e3 service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:00.254367 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14094fb3-9246-4348-a7bc-30892b3b19e3 service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:00.254882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14094fb3-9246-4348-a7bc-30892b3b19e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:00.258206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-14094fb3-9246-4348-a7bc-30892b3b19e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:00.384583 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14094fb3-9246-4348-a7bc-30892b3b19e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:38:00.387806 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14094fb3-9246-4348-a7bc-30892b3b19e3 service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:38:00.408054 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1145/2282] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:38:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:38:06.940034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7af9a8-faf3-432f-a9bd-2f9c1d4632ab service placement] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:38:07.230323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7af9a8-faf3-432f-a9bd-2f9c1d4632ab service placement] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:38:07.333504 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc6120-d57c-44c6-b6a1-33310d5f2b8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:38:07.336033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc6120-d57c-44c6-b6a1-33310d5f2b8e service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:07.336033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc6120-d57c-44c6-b6a1-33310d5f2b8e service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:07.336033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc6120-d57c-44c6-b6a1-33310d5f2b8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:07.338449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a8fc6120-d57c-44c6-b6a1-33310d5f2b8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:07.352978 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc6120-d57c-44c6-b6a1-33310d5f2b8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:38:07.355756 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc6120-d57c-44c6-b6a1-33310d5f2b8e service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:38:07.380340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1138/2283] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:38:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:38:08.585773 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd3875-194e-4b84-88bf-5d017d6b1f63 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:08.586662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd3875-194e-4b84-88bf-5d017d6b1f63 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:08.586921 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd3875-194e-4b84-88bf-5d017d6b1f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:08.637383 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b8dd3875-194e-4b84-88bf-5d017d6b1f63 None None] MFA Rules not processed for user `1b7d4a431f1c4feca7d33fcdb8ec1b7b`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:38:08.639925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b8dd3875-194e-4b84-88bf-5d017d6b1f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:08.816940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1146/2284] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:38:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:38:08.848597 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc6120-d57c-44c6-b6a1-33310d5f2b8e service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:38:08.850453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a8fc6120-d57c-44c6-b6a1-33310d5f2b8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:08.889629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc6120-d57c-44c6-b6a1-33310d5f2b8e service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:38:08.911352 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d1f06-f8d4-4512-b67d-9937c4a1a62e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:38:08.911352 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d1f06-f8d4-4512-b67d-9937c4a1a62e service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:08.911352 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d1f06-f8d4-4512-b67d-9937c4a1a62e service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:08.912521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d1f06-f8d4-4512-b67d-9937c4a1a62e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:09.077628 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d1f06-f8d4-4512-b67d-9937c4a1a62e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:38:09.086717 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d1f06-f8d4-4512-b67d-9937c4a1a62e service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:38:09.114178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1139/2285] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:38:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:38:13.260004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd3875-194e-4b84-88bf-5d017d6b1f63 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:38:13.305724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd3875-194e-4b84-88bf-5d017d6b1f63 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:38:13.322888 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce3445-eccf-49e6-931c-d2dbdce9b6a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:38:13.326009 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce3445-eccf-49e6-931c-d2dbdce9b6a4 service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:13.326009 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce3445-eccf-49e6-931c-d2dbdce9b6a4 service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:13.326009 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce3445-eccf-49e6-931c-d2dbdce9b6a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:13.333970 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-abce3445-eccf-49e6-931c-d2dbdce9b6a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:13.421633 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:38:13.466132 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce3445-eccf-49e6-931c-d2dbdce9b6a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:38:13.483936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce3445-eccf-49e6-931c-d2dbdce9b6a4 service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:38:13.523692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1147/2286] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:38:13 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:38:17.667514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d1f06-f8d4-4512-b67d-9937c4a1a62e service nova] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:38:17.670207 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6c1d1f06-f8d4-4512-b67d-9937c4a1a62e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:17.796429 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:38:17.806934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d1f06-f8d4-4512-b67d-9937c4a1a62e service nova] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:38:17.816483 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f3539-6ce8-4c9b-a396-467300244f6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:38:17.817940 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f3539-6ce8-4c9b-a396-467300244f6f service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:17.818651 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f3539-6ce8-4c9b-a396-467300244f6f service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:17.819380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f3539-6ce8-4c9b-a396-467300244f6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:17.823150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9f4f3539-6ce8-4c9b-a396-467300244f6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:17.921979 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f3539-6ce8-4c9b-a396-467300244f6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:38:17.924187 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f3539-6ce8-4c9b-a396-467300244f6f service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:38:17.943519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1140/2287] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:38:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:38:23.542129 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=9 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:38:27.934296 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=27 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:38:31.156302 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1c2ba-fe93-4e3b-97ef-36e7ef6f1c68 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:31.157166 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1c2ba-fe93-4e3b-97ef-36e7ef6f1c68 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:31.157166 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1c2ba-fe93-4e3b-97ef-36e7ef6f1c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:31.180790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-f2b1c2ba-fe93-4e3b-97ef-36e7ef6f1c68 None None] MFA Rules not processed for user `7e7db29f64934026abbe96d5cb5baf25`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:38:31.182073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f2b1c2ba-fe93-4e3b-97ef-36e7ef6f1c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:31.191082 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1148/2288] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:38:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:38:31.602795 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f3539-6ce8-4c9b-a396-467300244f6f service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:38:31.604623 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9f4f3539-6ce8-4c9b-a396-467300244f6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:31.619462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f3539-6ce8-4c9b-a396-467300244f6f service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:38:31.624295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f501bc0-1870-4236-b420-a14ac52a7af9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:38:31.624932 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f501bc0-1870-4236-b420-a14ac52a7af9 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:31.625218 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f501bc0-1870-4236-b420-a14ac52a7af9 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:31.625488 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f501bc0-1870-4236-b420-a14ac52a7af9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:31.629163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-8f501bc0-1870-4236-b420-a14ac52a7af9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:31.641838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f501bc0-1870-4236-b420-a14ac52a7af9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:38:31.643750 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f501bc0-1870-4236-b420-a14ac52a7af9 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:38:31.658687 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1141/2289] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:38:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:38:35.483624 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c7b94-65b2-406a-a28e-3708d044062e None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:35.484381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c7b94-65b2-406a-a28e-3708d044062e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:35.484381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c7b94-65b2-406a-a28e-3708d044062e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:35.506831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-d02c7b94-65b2-406a-a28e-3708d044062e None None] MFA Rules not processed for user `9dae215cd92f46d18a3c1b7389c3af4b`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:38:35.508129 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d02c7b94-65b2-406a-a28e-3708d044062e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:35.516114 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1149/2290] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:38:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:38:38.827352 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f501bc0-1870-4236-b420-a14ac52a7af9 service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:38:38.843243 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f501bc0-1870-4236-b420-a14ac52a7af9 service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:38:38.852562 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df7227-6b5b-4664-a86f-846f25999ab8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:38:38.853207 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df7227-6b5b-4664-a86f-846f25999ab8 service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:38.853493 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df7227-6b5b-4664-a86f-846f25999ab8 service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:38.853782 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df7227-6b5b-4664-a86f-846f25999ab8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:38.856081 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a8df7227-6b5b-4664-a86f-846f25999ab8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:38.873185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8df7227-6b5b-4664-a86f-846f25999ab8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:38:38.875092 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8df7227-6b5b-4664-a86f-846f25999ab8 service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:38:38.921110 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1142/2291] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:38:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:38:45.499856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=6 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:38:46.299998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c7b94-65b2-406a-a28e-3708d044062e None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:38:46.390160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c7b94-65b2-406a-a28e-3708d044062e None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:38:46.396116 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1886cd-71d5-46a8-98a1-8cdefc9eb61a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:38:46.397022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1886cd-71d5-46a8-98a1-8cdefc9eb61a service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:46.397304 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1886cd-71d5-46a8-98a1-8cdefc9eb61a service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:46.397578 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1886cd-71d5-46a8-98a1-8cdefc9eb61a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:46.400070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9d1886cd-71d5-46a8-98a1-8cdefc9eb61a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:46.457461 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1886cd-71d5-46a8-98a1-8cdefc9eb61a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:38:46.462379 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1886cd-71d5-46a8-98a1-8cdefc9eb61a service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:38:46.497090 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1150/2292] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:38:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:38:48.901770 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=12 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:38:50.927348 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df7227-6b5b-4664-a86f-846f25999ab8 service nova] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:38:51.027677 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df7227-6b5b-4664-a86f-846f25999ab8 service nova] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:38:51.040917 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e6c06-0f26-4f05-8066-362c6021629f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:38:51.042009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e6c06-0f26-4f05-8066-362c6021629f service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:51.042523 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e6c06-0f26-4f05-8066-362c6021629f service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:51.043078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e6c06-0f26-4f05-8066-362c6021629f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:51.046276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ab1e6c06-0f26-4f05-8066-362c6021629f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:51.068317 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e6c06-0f26-4f05-8066-362c6021629f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:38:51.070194 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e6c06-0f26-4f05-8066-362c6021629f service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:38:51.101936 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1143/2293] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:38:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:38:54.701565 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1886cd-71d5-46a8-98a1-8cdefc9eb61a service nova] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:38:54.842491 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1886cd-71d5-46a8-98a1-8cdefc9eb61a service nova] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:38:54.855992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72faf3-d5fe-4fb3-81e5-cb19bb8f7ca1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bee90bf951f04c2e8e19ceeacb8cfcb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:38:54.857057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72faf3-d5fe-4fb3-81e5-cb19bb8f7ca1 service placement] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:54.857779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72faf3-d5fe-4fb3-81e5-cb19bb8f7ca1 service placement] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:54.858444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72faf3-d5fe-4fb3-81e5-cb19bb8f7ca1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:54.862897 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-dc72faf3-d5fe-4fb3-81e5-cb19bb8f7ca1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:54.888669 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72faf3-d5fe-4fb3-81e5-cb19bb8f7ca1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:38:54.892161 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72faf3-d5fe-4fb3-81e5-cb19bb8f7ca1 service placement] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:38:54.914460 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1151/2294] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:38:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:38:55.237901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e6c06-0f26-4f05-8066-362c6021629f service nova] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:38:55.239603 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ab1e6c06-0f26-4f05-8066-362c6021629f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:38:55.379892 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e6c06-0f26-4f05-8066-362c6021629f service nova] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:38:55.385472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9aa43-2f64-4274-806c-376b7e43a32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:38:55.386502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9aa43-2f64-4274-806c-376b7e43a32b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:55.387184 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9aa43-2f64-4274-806c-376b7e43a32b admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:55.387690 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9aa43-2f64-4274-806c-376b7e43a32b admin admin] PATH_INFO: `/v3/users/6dcde357754a4f668c30c40dc838d09f` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:55.389846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc9aa43-2f64-4274-806c-376b7e43a32b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dcde357754a4f668c30c40dc838d09f)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:38:55.394277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc9aa43-2f64-4274-806c-376b7e43a32b admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:38:55.658805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5fc9aa43-2f64-4274-806c-376b7e43a32b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dcde357754a4f668c30c40dc838d09f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:38:55.677492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5fc9aa43-2f64-4274-806c-376b7e43a32b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dcde357754a4f668c30c40dc838d09f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:38:55.678287 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5fc9aa43-2f64-4274-806c-376b7e43a32b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dcde357754a4f668c30c40dc838d09f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:38:55.693315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5fc9aa43-2f64-4274-806c-376b7e43a32b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dcde357754a4f668c30c40dc838d09f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:38:55.704528 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5fc9aa43-2f64-4274-806c-376b7e43a32b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dcde357754a4f668c30c40dc838d09f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:38:55.827769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1144/2295] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:38:55 2022] DELETE /identity/v3/users/6dcde357754a4f668c30c40dc838d09f => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:38:56.233412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72faf3-d5fe-4fb3-81e5-cb19bb8f7ca1 service placement] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:38:56.266338 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72faf3-d5fe-4fb3-81e5-cb19bb8f7ca1 service placement] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:38:56.425845 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b1921-9823-44b1-afd8-459839a40b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:38:56.426860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b1921-9823-44b1-afd8-459839a40b7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:38:56.427354 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b1921-9823-44b1-afd8-459839a40b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:38:56.427844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b1921-9823-44b1-afd8-459839a40b7a admin admin] PATH_INFO: `/v3/projects/e0b6de13e7c64cbdb0f43bddfaebc4ba` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:38:56.430265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b1921-9823-44b1-afd8-459839a40b7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0b6de13e7c64cbdb0f43bddfaebc4ba)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:38:56.434459 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b1921-9823-44b1-afd8-459839a40b7a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:38:56.668374 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:38:56.686111 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-cd5b1921-9823-44b1-afd8-459839a40b7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0b6de13e7c64cbdb0f43bddfaebc4ba', 'request_id': 'req-cd5b1921-9823-44b1-afd8-459839a40b7a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:38:56.696846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-cd5b1921-9823-44b1-afd8-459839a40b7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0b6de13e7c64cbdb0f43bddfaebc4ba', 'request_id': 'req-cd5b1921-9823-44b1-afd8-459839a40b7a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:38:56.847283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-cd5b1921-9823-44b1-afd8-459839a40b7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0b6de13e7c64cbdb0f43bddfaebc4ba', 'request_id': 'req-cd5b1921-9823-44b1-afd8-459839a40b7a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:38:56.870640 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-cd5b1921-9823-44b1-afd8-459839a40b7a admin admin] The token cache is being invalidate because project e0b6de13e7c64cbdb0f43bddfaebc4ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:38:56.871302 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-cd5b1921-9823-44b1-afd8-459839a40b7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:38:56.872501 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-cd5b1921-9823-44b1-afd8-459839a40b7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:38:56.875858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1152/2296] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:38:56 2022] DELETE /identity/v3/projects/e0b6de13e7c64cbdb0f43bddfaebc4ba => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:05.823549 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:07.297956 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=13,keystone:DELETE=5,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:07.965154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9aa43-2f64-4274-806c-376b7e43a32b admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:07.975402 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5fc9aa43-2f64-4274-806c-376b7e43a32b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:08.131025 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9aa43-2f64-4274-806c-376b7e43a32b admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:08.230039 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a6929-1434-4f08-ad21-8fcf3a7aac87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:08.231288 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a6929-1434-4f08-ad21-8fcf3a7aac87 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:08.231842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a6929-1434-4f08-ad21-8fcf3a7aac87 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:08.232410 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a6929-1434-4f08-ad21-8fcf3a7aac87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:08.236565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d65a6929-1434-4f08-ad21-8fcf3a7aac87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:08.345966 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a6929-1434-4f08-ad21-8fcf3a7aac87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:08.347304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a6929-1434-4f08-ad21-8fcf3a7aac87 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:08.359220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1145/2297] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:39:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:39:08.395946 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b1921-9823-44b1-afd8-459839a40b7a admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:08.397951 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-cd5b1921-9823-44b1-afd8-459839a40b7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:08.417374 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b1921-9823-44b1-afd8-459839a40b7a admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:08.424189 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130ff98-c706-4d25-9bf9-c6a731ca20b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:08.425438 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130ff98-c706-4d25-9bf9-c6a731ca20b9 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:08.426011 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130ff98-c706-4d25-9bf9-c6a731ca20b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:08.426802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130ff98-c706-4d25-9bf9-c6a731ca20b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:08.430542 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b130ff98-c706-4d25-9bf9-c6a731ca20b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:08.568378 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130ff98-c706-4d25-9bf9-c6a731ca20b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:08.571493 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130ff98-c706-4d25-9bf9-c6a731ca20b9 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:08.591957 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1153/2298] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:39:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:39:14.865257 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a6929-1434-4f08-ad21-8fcf3a7aac87 service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:14.867606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d65a6929-1434-4f08-ad21-8fcf3a7aac87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:14.893130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a6929-1434-4f08-ad21-8fcf3a7aac87 service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:14.902311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44c4ff-4bbe-45a1-89ba-bde3bf3aef07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:14.906722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44c4ff-4bbe-45a1-89ba-bde3bf3aef07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:14.907300 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44c4ff-4bbe-45a1-89ba-bde3bf3aef07 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:14.907785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44c4ff-4bbe-45a1-89ba-bde3bf3aef07 admin admin] PATH_INFO: `/v3/users/a145e63a44db43acafc92bb2a238f463` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:14.910304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44c4ff-4bbe-45a1-89ba-bde3bf3aef07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a145e63a44db43acafc92bb2a238f463)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:14.914118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44c4ff-4bbe-45a1-89ba-bde3bf3aef07 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:15.229453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:15.236608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-da44c4ff-4bbe-45a1-89ba-bde3bf3aef07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a145e63a44db43acafc92bb2a238f463'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:15.256755 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-da44c4ff-4bbe-45a1-89ba-bde3bf3aef07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a145e63a44db43acafc92bb2a238f463'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:15.257746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-da44c4ff-4bbe-45a1-89ba-bde3bf3aef07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a145e63a44db43acafc92bb2a238f463'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:15.269180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-da44c4ff-4bbe-45a1-89ba-bde3bf3aef07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a145e63a44db43acafc92bb2a238f463'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:15.270364 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-da44c4ff-4bbe-45a1-89ba-bde3bf3aef07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a145e63a44db43acafc92bb2a238f463'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:15.391315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1146/2299] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:39:14 2022] DELETE /identity/v3/users/a145e63a44db43acafc92bb2a238f463 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:15.710588 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130ff98-c706-4d25-9bf9-c6a731ca20b9 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:15.731958 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130ff98-c706-4d25-9bf9-c6a731ca20b9 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:15.848196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94879510-0938-4d1e-9d1e-53e32f81d551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:15.848893 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94879510-0938-4d1e-9d1e-53e32f81d551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:15.849362 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94879510-0938-4d1e-9d1e-53e32f81d551 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:15.849798 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94879510-0938-4d1e-9d1e-53e32f81d551 admin admin] PATH_INFO: `/v3/projects/1070aeddb0d544f4b3b4c1e41979944d` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:15.851221 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94879510-0938-4d1e-9d1e-53e32f81d551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1070aeddb0d544f4b3b4c1e41979944d)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:15.853496 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94879510-0938-4d1e-9d1e-53e32f81d551 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:16.014143 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-94879510-0938-4d1e-9d1e-53e32f81d551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1070aeddb0d544f4b3b4c1e41979944d', 'request_id': 'req-94879510-0938-4d1e-9d1e-53e32f81d551'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:16.022626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-94879510-0938-4d1e-9d1e-53e32f81d551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1070aeddb0d544f4b3b4c1e41979944d', 'request_id': 'req-94879510-0938-4d1e-9d1e-53e32f81d551'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:16.151357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-94879510-0938-4d1e-9d1e-53e32f81d551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1070aeddb0d544f4b3b4c1e41979944d', 'request_id': 'req-94879510-0938-4d1e-9d1e-53e32f81d551'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:16.166005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-94879510-0938-4d1e-9d1e-53e32f81d551 admin admin] The token cache is being invalidate because project 1070aeddb0d544f4b3b4c1e41979944d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:39:16.166283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-94879510-0938-4d1e-9d1e-53e32f81d551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:16.166950 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-94879510-0938-4d1e-9d1e-53e32f81d551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:16.168495 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1154/2300] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:39:15 2022] DELETE /identity/v3/projects/1070aeddb0d544f4b3b4c1e41979944d => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:16.174664 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2aba6-0628-482e-90a7-558eb81bf00b None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:16.176713 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2aba6-0628-482e-90a7-558eb81bf00b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:16.176713 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2aba6-0628-482e-90a7-558eb81bf00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:16.285109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-3cb2aba6-0628-482e-90a7-558eb81bf00b None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:16.287933 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-3cb2aba6-0628-482e-90a7-558eb81bf00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:16.391763 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1147/2301] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:39:16.402860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593350a8-adf3-4378-9e62-ff09da59dbb8 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:16.403261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593350a8-adf3-4378-9e62-ff09da59dbb8 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:16.403716 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593350a8-adf3-4378-9e62-ff09da59dbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:16.429663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-593350a8-adf3-4378-9e62-ff09da59dbb8 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:16.430846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-593350a8-adf3-4378-9e62-ff09da59dbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:16.445290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1155/2302] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:39:16.453431 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2aba6-0628-482e-90a7-558eb81bf00b None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:16.472377 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2aba6-0628-482e-90a7-558eb81bf00b None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:16.481894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649907ab-3d39-4299-a37d-18d19e894c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:16.483038 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649907ab-3d39-4299-a37d-18d19e894c0a admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:16.483506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649907ab-3d39-4299-a37d-18d19e894c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:16.484480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649907ab-3d39-4299-a37d-18d19e894c0a admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:16.485374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649907ab-3d39-4299-a37d-18d19e894c0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:39:16.485850 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649907ab-3d39-4299-a37d-18d19e894c0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:16.488620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649907ab-3d39-4299-a37d-18d19e894c0a admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:16.505362 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1148/2303] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:39:16.510111 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593350a8-adf3-4378-9e62-ff09da59dbb8 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:16.521046 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593350a8-adf3-4378-9e62-ff09da59dbb8 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:16.526801 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a6c9b-e117-4f6e-ae3a-bcc518c1355a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:16.527647 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a6c9b-e117-4f6e-ae3a-bcc518c1355a admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:16.527926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a6c9b-e117-4f6e-ae3a-bcc518c1355a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:16.528203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a6c9b-e117-4f6e-ae3a-bcc518c1355a admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:16.528875 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a6c9b-e117-4f6e-ae3a-bcc518c1355a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:16.531006 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a6c9b-e117-4f6e-ae3a-bcc518c1355a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:16.551418 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:16.666807 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1156/2304] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:39:16 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:39:16.672832 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649907ab-3d39-4299-a37d-18d19e894c0a admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:16.685463 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649907ab-3d39-4299-a37d-18d19e894c0a admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:16.773391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151149e-0b41-4c4c-bfc8-777baa18293f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:16.774037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151149e-0b41-4c4c-bfc8-777baa18293f admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:16.774314 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151149e-0b41-4c4c-bfc8-777baa18293f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:16.774588 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151149e-0b41-4c4c-bfc8-777baa18293f admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:16.775295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151149e-0b41-4c4c-bfc8-777baa18293f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:16.777285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151149e-0b41-4c4c-bfc8-777baa18293f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:16.893592 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1149/2305] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:39:16 2022] POST /identity/v3/users => generated 286 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:39:16.898558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a6c9b-e117-4f6e-ae3a-bcc518c1355a admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:16.912401 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a6c9b-e117-4f6e-ae3a-bcc518c1355a admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:16.922679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29860fcc-84ac-4df6-a750-af8a0fd5c8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:16.923549 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29860fcc-84ac-4df6-a750-af8a0fd5c8cf admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:16.924082 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29860fcc-84ac-4df6-a750-af8a0fd5c8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:16.924398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29860fcc-84ac-4df6-a750-af8a0fd5c8cf admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:16.925216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29860fcc-84ac-4df6-a750-af8a0fd5c8cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:16.927133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29860fcc-84ac-4df6-a750-af8a0fd5c8cf admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:16.948238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1157/2306] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:39:16.956726 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151149e-0b41-4c4c-bfc8-777baa18293f admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:16.969066 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:16.977166 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151149e-0b41-4c4c-bfc8-777baa18293f admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:16.986120 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f20eb-db26-4c1c-ae85-83660ed91177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:16.987107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f20eb-db26-4c1c-ae85-83660ed91177 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:16.987393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f20eb-db26-4c1c-ae85-83660ed91177 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:16.987829 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f20eb-db26-4c1c-ae85-83660ed91177 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:16.988937 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f20eb-db26-4c1c-ae85-83660ed91177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:16.991569 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f20eb-db26-4c1c-ae85-83660ed91177 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:17.013758 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1150/2307] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:39:17.016383 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29860fcc-84ac-4df6-a750-af8a0fd5c8cf admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:17.042607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29860fcc-84ac-4df6-a750-af8a0fd5c8cf admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:17.049408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665bdb5-ff86-4baa-a905-3ae6d9a50135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:17.050097 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665bdb5-ff86-4baa-a905-3ae6d9a50135 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:17.050382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665bdb5-ff86-4baa-a905-3ae6d9a50135 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:17.050657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665bdb5-ff86-4baa-a905-3ae6d9a50135 admin admin] PATH_INFO: `/v3/projects/a03c36a97902448684a0ea4cb1034bef/users/a4e0972663f54218a2ba7c2ad2d9aa2f/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:17.071077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e665bdb5-ff86-4baa-a905-3ae6d9a50135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a03c36a97902448684a0ea4cb1034bef, user_id=a4e0972663f54218a2ba7c2ad2d9aa2f, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:17.073924 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e665bdb5-ff86-4baa-a905-3ae6d9a50135 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:17.205863 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1158/2308] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:39:17 2022] PUT /identity/v3/projects/a03c36a97902448684a0ea4cb1034bef/users/a4e0972663f54218a2ba7c2ad2d9aa2f/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:17.217958 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcc020-1d82-4e8a-a26d-27a8eba737a5 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:17.217958 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcc020-1d82-4e8a-a26d-27a8eba737a5 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:17.220194 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcc020-1d82-4e8a-a26d-27a8eba737a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:17.256967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-efdcc020-1d82-4e8a-a26d-27a8eba737a5 None None] MFA Rules not processed for user `a4e0972663f54218a2ba7c2ad2d9aa2f`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:17.258736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-efdcc020-1d82-4e8a-a26d-27a8eba737a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:17.344035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1151/2309] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:39:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:39:17.349048 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665bdb5-ff86-4baa-a905-3ae6d9a50135 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:17.444801 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665bdb5-ff86-4baa-a905-3ae6d9a50135 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:17.450110 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d7cd3-d8a6-4bd7-afc8-4a8b3967f837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:17.450796 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7cd3-d8a6-4bd7-afc8-4a8b3967f837 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:17.451159 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7cd3-d8a6-4bd7-afc8-4a8b3967f837 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:17.451474 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7cd3-d8a6-4bd7-afc8-4a8b3967f837 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:17.453732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d7cd3-d8a6-4bd7-afc8-4a8b3967f837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:17.456074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d7cd3-d8a6-4bd7-afc8-4a8b3967f837 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:17.576553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1159/2310] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:39:17 2022] POST /identity/v3/projects => generated 373 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:39:17.584818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcc020-1d82-4e8a-a26d-27a8eba737a5 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:17.599412 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcc020-1d82-4e8a-a26d-27a8eba737a5 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:17.672883 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7aa1a5-c3f3-4935-94c8-c7a21232e070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:17.673525 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7aa1a5-c3f3-4935-94c8-c7a21232e070 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:17.673812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7aa1a5-c3f3-4935-94c8-c7a21232e070 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:17.674089 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7aa1a5-c3f3-4935-94c8-c7a21232e070 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:17.674748 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7aa1a5-c3f3-4935-94c8-c7a21232e070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:17.676831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7aa1a5-c3f3-4935-94c8-c7a21232e070 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:17.809522 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1152/2311] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:39:17 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:39:17.815511 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d7cd3-d8a6-4bd7-afc8-4a8b3967f837 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:17.832754 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d7cd3-d8a6-4bd7-afc8-4a8b3967f837 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:17.844087 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70880cb8-ccd4-491c-98cb-26417d5156d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:17.845318 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70880cb8-ccd4-491c-98cb-26417d5156d3 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:17.846034 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70880cb8-ccd4-491c-98cb-26417d5156d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:17.846909 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70880cb8-ccd4-491c-98cb-26417d5156d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:17.848247 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70880cb8-ccd4-491c-98cb-26417d5156d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:17.851557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70880cb8-ccd4-491c-98cb-26417d5156d3 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:17.879117 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1160/2312] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:39:17.881140 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7aa1a5-c3f3-4935-94c8-c7a21232e070 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:17.892589 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7aa1a5-c3f3-4935-94c8-c7a21232e070 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:17.899966 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab114ec1-cfab-4178-a62d-8b0846b8c2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:17.900836 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab114ec1-cfab-4178-a62d-8b0846b8c2ae admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:17.901233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab114ec1-cfab-4178-a62d-8b0846b8c2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:17.902358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab114ec1-cfab-4178-a62d-8b0846b8c2ae admin admin] PATH_INFO: `/v3/projects/84b4e1332aa3424983b474de289f410f/users/0b8d13154e4a43efa5cf397a654185e7/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:17.920967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab114ec1-cfab-4178-a62d-8b0846b8c2ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b4e1332aa3424983b474de289f410f, user_id=0b8d13154e4a43efa5cf397a654185e7, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:17.925003 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab114ec1-cfab-4178-a62d-8b0846b8c2ae admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:18.033500 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1153/2313] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:39:17 2022] PUT /identity/v3/projects/84b4e1332aa3424983b474de289f410f/users/0b8d13154e4a43efa5cf397a654185e7/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:18.038692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70880cb8-ccd4-491c-98cb-26417d5156d3 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:18.051650 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70880cb8-ccd4-491c-98cb-26417d5156d3 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:18.124708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10eaa1-a829-405f-b1ed-f5638873afbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:18.125508 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10eaa1-a829-405f-b1ed-f5638873afbf admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:18.125944 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10eaa1-a829-405f-b1ed-f5638873afbf admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:18.126232 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10eaa1-a829-405f-b1ed-f5638873afbf admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:18.126952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10eaa1-a829-405f-b1ed-f5638873afbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:18.128704 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10eaa1-a829-405f-b1ed-f5638873afbf admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:18.141977 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1161/2314] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:39:18.147222 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab114ec1-cfab-4178-a62d-8b0846b8c2ae admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:18.160082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab114ec1-cfab-4178-a62d-8b0846b8c2ae admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:18.166009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e6320-3c6c-478c-ba02-1c5d2fc40d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:18.166701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e6320-3c6c-478c-ba02-1c5d2fc40d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:18.167009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e6320-3c6c-478c-ba02-1c5d2fc40d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:18.167298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e6320-3c6c-478c-ba02-1c5d2fc40d39 admin admin] PATH_INFO: `/v3/projects/84b4e1332aa3424983b474de289f410f/users/0b8d13154e4a43efa5cf397a654185e7/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:18.169450 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e6320-3c6c-478c-ba02-1c5d2fc40d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b4e1332aa3424983b474de289f410f, user_id=0b8d13154e4a43efa5cf397a654185e7, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:18.172337 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e6320-3c6c-478c-ba02-1c5d2fc40d39 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:18.185578 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1154/2315] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:39:18 2022] PUT /identity/v3/projects/84b4e1332aa3424983b474de289f410f/users/0b8d13154e4a43efa5cf397a654185e7/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:18.198042 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a5d69-9d6a-42a8-b1ea-1a3b31a3bd3d None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:18.198466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a5d69-9d6a-42a8-b1ea-1a3b31a3bd3d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:18.198833 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a5d69-9d6a-42a8-b1ea-1a3b31a3bd3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:18.236789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-9e1a5d69-9d6a-42a8-b1ea-1a3b31a3bd3d None None] MFA Rules not processed for user `0b8d13154e4a43efa5cf397a654185e7`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:18.238375 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9e1a5d69-9d6a-42a8-b1ea-1a3b31a3bd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:18.251100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:18.348254 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1162/2316] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:39:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:39:18.362673 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e6320-3c6c-478c-ba02-1c5d2fc40d39 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:18.364742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-937e6320-3c6c-478c-ba02-1c5d2fc40d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:18.416104 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:18.469049 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e6320-3c6c-478c-ba02-1c5d2fc40d39 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:18.476013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ddff0-a054-4438-92bb-f68ba3d8ffaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:18.477189 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ddff0-a054-4438-92bb-f68ba3d8ffaa service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:18.477843 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ddff0-a054-4438-92bb-f68ba3d8ffaa service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:18.478403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ddff0-a054-4438-92bb-f68ba3d8ffaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:18.498045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ddff0-a054-4438-92bb-f68ba3d8ffaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:18.502016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ddff0-a054-4438-92bb-f68ba3d8ffaa service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:18.522535 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1155/2317] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:39:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:39:19.998950 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a5d69-9d6a-42a8-b1ea-1a3b31a3bd3d None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:20.083715 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a5d69-9d6a-42a8-b1ea-1a3b31a3bd3d None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:20.088884 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa272a8d-4c36-49c0-a6b0-e322ccdf1bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:20.089558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa272a8d-4c36-49c0-a6b0-e322ccdf1bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:20.089850 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa272a8d-4c36-49c0-a6b0-e322ccdf1bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:20.090132 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa272a8d-4c36-49c0-a6b0-e322ccdf1bcc admin admin] PATH_INFO: `/v3/users/a4e0972663f54218a2ba7c2ad2d9aa2f` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:20.091504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa272a8d-4c36-49c0-a6b0-e322ccdf1bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4e0972663f54218a2ba7c2ad2d9aa2f)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:20.094756 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa272a8d-4c36-49c0-a6b0-e322ccdf1bcc admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:20.342605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-fa272a8d-4c36-49c0-a6b0-e322ccdf1bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4e0972663f54218a2ba7c2ad2d9aa2f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:20.442394 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-fa272a8d-4c36-49c0-a6b0-e322ccdf1bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4e0972663f54218a2ba7c2ad2d9aa2f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:20.443082 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-fa272a8d-4c36-49c0-a6b0-e322ccdf1bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4e0972663f54218a2ba7c2ad2d9aa2f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:20.443850 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-fa272a8d-4c36-49c0-a6b0-e322ccdf1bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4e0972663f54218a2ba7c2ad2d9aa2f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:20.454412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-fa272a8d-4c36-49c0-a6b0-e322ccdf1bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4e0972663f54218a2ba7c2ad2d9aa2f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:20.488300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1163/2318] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:39:19 2022] DELETE /identity/v3/users/a4e0972663f54218a2ba7c2ad2d9aa2f => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:39:20.490594 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb38c1-78b4-4821-943a-add89a2f94e5 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:20.491398 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb38c1-78b4-4821-943a-add89a2f94e5 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:20.491398 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb38c1-78b4-4821-943a-add89a2f94e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:20.516809 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-e4bb38c1-78b4-4821-943a-add89a2f94e5 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:20.518199 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e4bb38c1-78b4-4821-943a-add89a2f94e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:20.604885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1156/2319] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:39:20.617893 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa272a8d-4c36-49c0-a6b0-e322ccdf1bcc admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:20.619381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-fa272a8d-4c36-49c0-a6b0-e322ccdf1bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:20.666632 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:20.707824 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa272a8d-4c36-49c0-a6b0-e322ccdf1bcc admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:20.712959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27915bc1-e5ea-4e05-8b0c-8b601d6575b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:20.713675 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27915bc1-e5ea-4e05-8b0c-8b601d6575b1 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:20.714288 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27915bc1-e5ea-4e05-8b0c-8b601d6575b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:20.714564 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27915bc1-e5ea-4e05-8b0c-8b601d6575b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:20.729958 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27915bc1-e5ea-4e05-8b0c-8b601d6575b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:20.732851 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27915bc1-e5ea-4e05-8b0c-8b601d6575b1 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:20.751848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1164/2320] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:39:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:39:21.469365 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb38c1-78b4-4821-943a-add89a2f94e5 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:21.472200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e4bb38c1-78b4-4821-943a-add89a2f94e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:21.495476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb38c1-78b4-4821-943a-add89a2f94e5 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:21.510247 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41624383-1184-4585-a4ef-021c5ca2c75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:21.512255 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41624383-1184-4585-a4ef-021c5ca2c75f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:21.513846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41624383-1184-4585-a4ef-021c5ca2c75f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:21.514655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41624383-1184-4585-a4ef-021c5ca2c75f admin admin] PATH_INFO: `/v3/projects/a03c36a97902448684a0ea4cb1034bef` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:21.519926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41624383-1184-4585-a4ef-021c5ca2c75f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a03c36a97902448684a0ea4cb1034bef)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:21.525892 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41624383-1184-4585-a4ef-021c5ca2c75f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:21.815646 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-41624383-1184-4585-a4ef-021c5ca2c75f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a03c36a97902448684a0ea4cb1034bef', 'request_id': 'req-41624383-1184-4585-a4ef-021c5ca2c75f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:21.834819 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-41624383-1184-4585-a4ef-021c5ca2c75f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a03c36a97902448684a0ea4cb1034bef', 'request_id': 'req-41624383-1184-4585-a4ef-021c5ca2c75f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:22.031872 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-41624383-1184-4585-a4ef-021c5ca2c75f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a03c36a97902448684a0ea4cb1034bef', 'request_id': 'req-41624383-1184-4585-a4ef-021c5ca2c75f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:22.053774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-41624383-1184-4585-a4ef-021c5ca2c75f admin admin] The token cache is being invalidate because project a03c36a97902448684a0ea4cb1034bef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:39:22.054290 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-41624383-1184-4585-a4ef-021c5ca2c75f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:22.055478 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-41624383-1184-4585-a4ef-021c5ca2c75f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:22.058417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1157/2321] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:39:21 2022] DELETE /identity/v3/projects/a03c36a97902448684a0ea4cb1034bef => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:22.065583 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27915bc1-e5ea-4e05-8b0c-8b601d6575b1 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:22.089755 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27915bc1-e5ea-4e05-8b0c-8b601d6575b1 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:22.184028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f3a2b-da11-4740-8c2a-557949a10adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:22.184684 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f3a2b-da11-4740-8c2a-557949a10adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:22.185049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f3a2b-da11-4740-8c2a-557949a10adb admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:22.185254 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f3a2b-da11-4740-8c2a-557949a10adb admin admin] PATH_INFO: `/v3/users/0b8d13154e4a43efa5cf397a654185e7` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:22.186531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f3a2b-da11-4740-8c2a-557949a10adb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b8d13154e4a43efa5cf397a654185e7)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:22.189334 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f3a2b-da11-4740-8c2a-557949a10adb admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:22.444244 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-df8f3a2b-da11-4740-8c2a-557949a10adb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b8d13154e4a43efa5cf397a654185e7'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:22.539835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-df8f3a2b-da11-4740-8c2a-557949a10adb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b8d13154e4a43efa5cf397a654185e7'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:22.540880 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-df8f3a2b-da11-4740-8c2a-557949a10adb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b8d13154e4a43efa5cf397a654185e7'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:22.541760 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-df8f3a2b-da11-4740-8c2a-557949a10adb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b8d13154e4a43efa5cf397a654185e7'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:22.554672 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-df8f3a2b-da11-4740-8c2a-557949a10adb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b8d13154e4a43efa5cf397a654185e7'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:22.602915 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1165/2322] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:39:22 2022] DELETE /identity/v3/users/0b8d13154e4a43efa5cf397a654185e7 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:23.379679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41624383-1184-4585-a4ef-021c5ca2c75f admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:23.382172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-41624383-1184-4585-a4ef-021c5ca2c75f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:23.413584 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41624383-1184-4585-a4ef-021c5ca2c75f admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:23.486333 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:23.507674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a67498-a9ee-40c0-97b2-7383cc8618eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:23.508337 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a67498-a9ee-40c0-97b2-7383cc8618eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:23.508623 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a67498-a9ee-40c0-97b2-7383cc8618eb admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:23.509055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a67498-a9ee-40c0-97b2-7383cc8618eb admin admin] PATH_INFO: `/v3/projects/84b4e1332aa3424983b474de289f410f` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:23.510482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a67498-a9ee-40c0-97b2-7383cc8618eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=84b4e1332aa3424983b474de289f410f)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:23.512838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a67498-a9ee-40c0-97b2-7383cc8618eb admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:23.757571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a1a67498-a9ee-40c0-97b2-7383cc8618eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84b4e1332aa3424983b474de289f410f', 'request_id': 'req-a1a67498-a9ee-40c0-97b2-7383cc8618eb'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:23.766496 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a1a67498-a9ee-40c0-97b2-7383cc8618eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84b4e1332aa3424983b474de289f410f', 'request_id': 'req-a1a67498-a9ee-40c0-97b2-7383cc8618eb'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:23.936544 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a1a67498-a9ee-40c0-97b2-7383cc8618eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84b4e1332aa3424983b474de289f410f', 'request_id': 'req-a1a67498-a9ee-40c0-97b2-7383cc8618eb'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:23.953486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a1a67498-a9ee-40c0-97b2-7383cc8618eb admin admin] The token cache is being invalidate because project 84b4e1332aa3424983b474de289f410f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:39:23.962531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a1a67498-a9ee-40c0-97b2-7383cc8618eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:23.962531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a1a67498-a9ee-40c0-97b2-7383cc8618eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:23.962531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1158/2323] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:39:23 2022] DELETE /identity/v3/projects/84b4e1332aa3424983b474de289f410f => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:23.962968 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3fe29-92c2-4be2-b671-121e2fa3205c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:23.962968 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3fe29-92c2-4be2-b671-121e2fa3205c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:23.962968 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3fe29-92c2-4be2-b671-121e2fa3205c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:23.984362 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-34a3fe29-92c2-4be2-b671-121e2fa3205c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:23.985694 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-34a3fe29-92c2-4be2-b671-121e2fa3205c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:24.039867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:24.071887 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1166/2324] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:39:24.088710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada6ce8-b288-43a0-aa52-c03f3793bc9e None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:24.088710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada6ce8-b288-43a0-aa52-c03f3793bc9e None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:24.088710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada6ce8-b288-43a0-aa52-c03f3793bc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:24.107201 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-aada6ce8-b288-43a0-aa52-c03f3793bc9e None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:24.109071 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-aada6ce8-b288-43a0-aa52-c03f3793bc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:24.118643 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1159/2325] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:39:24.124092 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3fe29-92c2-4be2-b671-121e2fa3205c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:24.141865 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3fe29-92c2-4be2-b671-121e2fa3205c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:24.159130 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be9159-f8cd-4678-be40-fd00d7308ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:24.159130 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be9159-f8cd-4678-be40-fd00d7308ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:24.159130 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be9159-f8cd-4678-be40-fd00d7308ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:24.159130 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be9159-f8cd-4678-be40-fd00d7308ec3 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:24.159930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be9159-f8cd-4678-be40-fd00d7308ec3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:39:24.159930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be9159-f8cd-4678-be40-fd00d7308ec3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:24.159930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be9159-f8cd-4678-be40-fd00d7308ec3 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:24.173423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1167/2326] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:39:24.176231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada6ce8-b288-43a0-aa52-c03f3793bc9e None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:24.197926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada6ce8-b288-43a0-aa52-c03f3793bc9e None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:24.206858 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cea6a3-00b5-44cd-9ec5-e155f63fa88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:24.211193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cea6a3-00b5-44cd-9ec5-e155f63fa88a admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:24.211193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cea6a3-00b5-44cd-9ec5-e155f63fa88a admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:24.211193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cea6a3-00b5-44cd-9ec5-e155f63fa88a admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:24.211193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cea6a3-00b5-44cd-9ec5-e155f63fa88a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:24.214443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cea6a3-00b5-44cd-9ec5-e155f63fa88a admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:24.370582 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1160/2327] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:39:24 2022] POST /identity/v3/projects => generated 409 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:39:24.377588 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be9159-f8cd-4678-be40-fd00d7308ec3 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:24.392986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be9159-f8cd-4678-be40-fd00d7308ec3 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:24.495681 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4435a9-da4a-4a11-88d2-ee9430e826f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:24.496415 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4435a9-da4a-4a11-88d2-ee9430e826f7 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:24.496783 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4435a9-da4a-4a11-88d2-ee9430e826f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:24.497132 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4435a9-da4a-4a11-88d2-ee9430e826f7 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:24.497811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4435a9-da4a-4a11-88d2-ee9430e826f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:24.500813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4435a9-da4a-4a11-88d2-ee9430e826f7 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:24.630731 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1168/2328] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:39:24 2022] POST /identity/v3/users => generated 305 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:39:24.636507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cea6a3-00b5-44cd-9ec5-e155f63fa88a admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:24.667142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cea6a3-00b5-44cd-9ec5-e155f63fa88a admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:24.674157 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fe468-9271-4acb-8fa8-09182b6763b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:24.675421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fe468-9271-4acb-8fa8-09182b6763b9 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:24.676111 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fe468-9271-4acb-8fa8-09182b6763b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:24.676516 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fe468-9271-4acb-8fa8-09182b6763b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:24.677061 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fe468-9271-4acb-8fa8-09182b6763b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:24.678785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fe468-9271-4acb-8fa8-09182b6763b9 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:24.693484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1161/2329] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:39:24.698344 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4435a9-da4a-4a11-88d2-ee9430e826f7 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:24.711241 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4435a9-da4a-4a11-88d2-ee9430e826f7 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:24.719004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4623c-d85f-4bde-b747-223516ada5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:24.719810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4623c-d85f-4bde-b747-223516ada5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:24.719960 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4623c-d85f-4bde-b747-223516ada5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:24.720281 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4623c-d85f-4bde-b747-223516ada5c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:24.721137 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee4623c-d85f-4bde-b747-223516ada5c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:24.722905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee4623c-d85f-4bde-b747-223516ada5c8 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:24.736621 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1169/2330] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:39:24.742532 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fe468-9271-4acb-8fa8-09182b6763b9 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:24.758059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fe468-9271-4acb-8fa8-09182b6763b9 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:24.763621 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ee365-ea4d-4361-9441-136e4555a222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:24.769314 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ee365-ea4d-4361-9441-136e4555a222 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:24.769314 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ee365-ea4d-4361-9441-136e4555a222 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:24.769314 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ee365-ea4d-4361-9441-136e4555a222 admin admin] PATH_INFO: `/v3/projects/8032bf2514c440fb8b28ca927bb568e3/users/eed51a45b96e45f5b0c7382e00996063/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:24.791150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ee365-ea4d-4361-9441-136e4555a222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8032bf2514c440fb8b28ca927bb568e3, user_id=eed51a45b96e45f5b0c7382e00996063, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:24.797090 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ee365-ea4d-4361-9441-136e4555a222 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:24.950959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1162/2331] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:39:24 2022] PUT /identity/v3/projects/8032bf2514c440fb8b28ca927bb568e3/users/eed51a45b96e45f5b0c7382e00996063/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:39:24.963054 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fe1dd-5765-4ed7-a351-c3fd411f9d52 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:24.963610 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fe1dd-5765-4ed7-a351-c3fd411f9d52 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:24.963855 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fe1dd-5765-4ed7-a351-c3fd411f9d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:25.000244 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-848fe1dd-5765-4ed7-a351-c3fd411f9d52 None None] MFA Rules not processed for user `eed51a45b96e45f5b0c7382e00996063`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:25.001797 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-848fe1dd-5765-4ed7-a351-c3fd411f9d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:25.089535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1170/2332] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:39:24 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:39:25.095262 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ee365-ea4d-4361-9441-136e4555a222 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:25.174400 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ee365-ea4d-4361-9441-136e4555a222 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:25.181753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f57ef-4088-4062-b43e-8a022748b491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:25.182879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f57ef-4088-4062-b43e-8a022748b491 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:25.183463 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f57ef-4088-4062-b43e-8a022748b491 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:25.183942 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f57ef-4088-4062-b43e-8a022748b491 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:25.185166 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f57ef-4088-4062-b43e-8a022748b491 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:25.189755 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f57ef-4088-4062-b43e-8a022748b491 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:25.219439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:25.371335 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1163/2333] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:39:25 2022] POST /identity/v3/projects => generated 409 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:39:25.377634 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fe1dd-5765-4ed7-a351-c3fd411f9d52 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:25.414318 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fe1dd-5765-4ed7-a351-c3fd411f9d52 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:25.500558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:25.529960 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17174876-a40f-4e82-bf5d-7afa4acd3064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:25.531522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17174876-a40f-4e82-bf5d-7afa4acd3064 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:25.532026 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17174876-a40f-4e82-bf5d-7afa4acd3064 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:25.532516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17174876-a40f-4e82-bf5d-7afa4acd3064 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:25.533416 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17174876-a40f-4e82-bf5d-7afa4acd3064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:25.536308 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17174876-a40f-4e82-bf5d-7afa4acd3064 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:25.670750 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1171/2334] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:39:25 2022] POST /identity/v3/users => generated 311 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:39:25.676280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f57ef-4088-4062-b43e-8a022748b491 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:25.696592 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f57ef-4088-4062-b43e-8a022748b491 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:25.706406 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08336336-5274-4fbd-b858-d46ce14d5b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:25.707514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336336-5274-4fbd-b858-d46ce14d5b6b admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:25.708185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336336-5274-4fbd-b858-d46ce14d5b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:25.708582 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336336-5274-4fbd-b858-d46ce14d5b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:25.709996 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08336336-5274-4fbd-b858-d46ce14d5b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:25.713276 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08336336-5274-4fbd-b858-d46ce14d5b6b admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:25.729846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1164/2335] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:39:25.738192 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17174876-a40f-4e82-bf5d-7afa4acd3064 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:25.758559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17174876-a40f-4e82-bf5d-7afa4acd3064 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:25.770762 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be58df6-6378-444d-a7a4-e6629f40fef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:25.772017 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be58df6-6378-444d-a7a4-e6629f40fef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:25.772627 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be58df6-6378-444d-a7a4-e6629f40fef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:25.773204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be58df6-6378-444d-a7a4-e6629f40fef1 admin admin] PATH_INFO: `/v3/projects/90de8d99b02e4d1b9e3f9d967f5a9256/users/9712c5b5ff4b40a49af079cd2f26b2b4/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:25.798558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be58df6-6378-444d-a7a4-e6629f40fef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90de8d99b02e4d1b9e3f9d967f5a9256, user_id=9712c5b5ff4b40a49af079cd2f26b2b4, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:25.801594 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be58df6-6378-444d-a7a4-e6629f40fef1 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:25.985083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1172/2336] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:39:25 2022] PUT /identity/v3/projects/90de8d99b02e4d1b9e3f9d967f5a9256/users/9712c5b5ff4b40a49af079cd2f26b2b4/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:25.990466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08336336-5274-4fbd-b858-d46ce14d5b6b admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:26.007357 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08336336-5274-4fbd-b858-d46ce14d5b6b admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:26.095112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340c775-e3da-4116-ac8f-01dc0dcaabdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:26.096238 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340c775-e3da-4116-ac8f-01dc0dcaabdc admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:26.096743 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340c775-e3da-4116-ac8f-01dc0dcaabdc admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:26.097244 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340c775-e3da-4116-ac8f-01dc0dcaabdc admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:26.098530 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340c775-e3da-4116-ac8f-01dc0dcaabdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:26.101891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340c775-e3da-4116-ac8f-01dc0dcaabdc admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:26.124347 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1165/2337] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:39:26.129584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be58df6-6378-444d-a7a4-e6629f40fef1 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:26.149165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be58df6-6378-444d-a7a4-e6629f40fef1 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:26.158536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5256476-08e5-4c51-8b61-b4d10653fa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:26.159760 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5256476-08e5-4c51-8b61-b4d10653fa09 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:26.160260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5256476-08e5-4c51-8b61-b4d10653fa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:26.160763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5256476-08e5-4c51-8b61-b4d10653fa09 admin admin] PATH_INFO: `/v3/projects/90de8d99b02e4d1b9e3f9d967f5a9256/users/9712c5b5ff4b40a49af079cd2f26b2b4/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:26.164973 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5256476-08e5-4c51-8b61-b4d10653fa09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90de8d99b02e4d1b9e3f9d967f5a9256, user_id=9712c5b5ff4b40a49af079cd2f26b2b4, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:26.170722 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5256476-08e5-4c51-8b61-b4d10653fa09 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:26.192688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1173/2338] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:39:26 2022] PUT /identity/v3/projects/90de8d99b02e4d1b9e3f9d967f5a9256/users/9712c5b5ff4b40a49af079cd2f26b2b4/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:39:26.204564 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5a06a-3e87-4ec5-8033-9900f7b7c575 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:26.204884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5a06a-3e87-4ec5-8033-9900f7b7c575 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:26.205172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5a06a-3e87-4ec5-8033-9900f7b7c575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:26.243158 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-d9f5a06a-3e87-4ec5-8033-9900f7b7c575 None None] MFA Rules not processed for user `9712c5b5ff4b40a49af079cd2f26b2b4`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:26.244716 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d9f5a06a-3e87-4ec5-8033-9900f7b7c575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:26.359013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1166/2339] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:39:26 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:39:26.374400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5256476-08e5-4c51-8b61-b4d10653fa09 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:26.377664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c5256476-08e5-4c51-8b61-b4d10653fa09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:26.461674 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5256476-08e5-4c51-8b61-b4d10653fa09 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:26.466358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc138d-0623-48da-b947-884d40d41022 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:26.467113 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc138d-0623-48da-b947-884d40d41022 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:26.467543 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc138d-0623-48da-b947-884d40d41022 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:26.468461 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc138d-0623-48da-b947-884d40d41022 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:26.480937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc138d-0623-48da-b947-884d40d41022 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:26.484014 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc138d-0623-48da-b947-884d40d41022 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:26.497650 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1174/2340] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:39:26 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:39:27.123021 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5a06a-3e87-4ec5-8033-9900f7b7c575 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:27.139255 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5a06a-3e87-4ec5-8033-9900f7b7c575 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:27.146085 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683d4bf-a736-429c-9032-3fcfc2bd3535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:27.147184 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683d4bf-a736-429c-9032-3fcfc2bd3535 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:27.147626 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683d4bf-a736-429c-9032-3fcfc2bd3535 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:27.148028 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683d4bf-a736-429c-9032-3fcfc2bd3535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:27.299912 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7683d4bf-a736-429c-9032-3fcfc2bd3535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:27.303324 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7683d4bf-a736-429c-9032-3fcfc2bd3535 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:27.325960 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1167/2341] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:39:27 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:39:29.017793 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26168314-ef6e-43e4-b862-b691fb205077 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:29.018425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26168314-ef6e-43e4-b862-b691fb205077 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:29.018740 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26168314-ef6e-43e4-b862-b691fb205077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:29.046299 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-26168314-ef6e-43e4-b862-b691fb205077 None None] MFA Rules not processed for user `c5e931aa22344990a0f032159ebc20ad`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:29.047749 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-26168314-ef6e-43e4-b862-b691fb205077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:29.123258 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1175/2342] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:39:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:39:29.605714 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683d4bf-a736-429c-9032-3fcfc2bd3535 service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:29.608418 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-7683d4bf-a736-429c-9032-3fcfc2bd3535 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:29.617688 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:29.701565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683d4bf-a736-429c-9032-3fcfc2bd3535 service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:29.706588 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94d863-e224-475a-aca4-ed2f91f39a05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:29.707984 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94d863-e224-475a-aca4-ed2f91f39a05 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:29.707984 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94d863-e224-475a-aca4-ed2f91f39a05 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:29.707984 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94d863-e224-475a-aca4-ed2f91f39a05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:29.710213 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ec94d863-e224-475a-aca4-ed2f91f39a05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:29.723430 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94d863-e224-475a-aca4-ed2f91f39a05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:29.728957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94d863-e224-475a-aca4-ed2f91f39a05 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:29.742231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1168/2343] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:39:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:39:34.270109 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d5840-c3a7-43fe-965e-f75d69f77f27 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:34.270713 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d5840-c3a7-43fe-965e-f75d69f77f27 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:34.270849 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d5840-c3a7-43fe-965e-f75d69f77f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:34.315498 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-0d4d5840-c3a7-43fe-965e-f75d69f77f27 None None] MFA Rules not processed for user `8121ba4cf2cf44d9a5a55975ea85928c`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:34.317280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0d4d5840-c3a7-43fe-965e-f75d69f77f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:34.393210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:34.449022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1176/2344] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 7 13:39:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:39:34.724329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94d863-e224-475a-aca4-ed2f91f39a05 service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:34.726883 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ec94d863-e224-475a-aca4-ed2f91f39a05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:34.821583 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94d863-e224-475a-aca4-ed2f91f39a05 service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:34.831611 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19116543-9223-41e7-bd41-12f97bcc93fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cedc7091d2f46b5be68818d26399e9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:34.831611 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19116543-9223-41e7-bd41-12f97bcc93fe service swift] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:34.831611 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19116543-9223-41e7-bd41-12f97bcc93fe service swift] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:34.832318 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19116543-9223-41e7-bd41-12f97bcc93fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:34.847740 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19116543-9223-41e7-bd41-12f97bcc93fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:34.849577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19116543-9223-41e7-bd41-12f97bcc93fe service swift] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:34.862440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1169/2345] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 7 13:39:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:39:39.714237 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d5840-c3a7-43fe-965e-f75d69f77f27 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:39.954158 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d5840-c3a7-43fe-965e-f75d69f77f27 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:39.972282 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168f890-f0a0-41f8-9ea4-a00ef3b9e541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:39.973693 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168f890-f0a0-41f8-9ea4-a00ef3b9e541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:39.974751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168f890-f0a0-41f8-9ea4-a00ef3b9e541 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:39.975403 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168f890-f0a0-41f8-9ea4-a00ef3b9e541 admin admin] PATH_INFO: `/v3/users/eed51a45b96e45f5b0c7382e00996063` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:39.978229 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b168f890-f0a0-41f8-9ea4-a00ef3b9e541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eed51a45b96e45f5b0c7382e00996063)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:39.982698 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b168f890-f0a0-41f8-9ea4-a00ef3b9e541 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:40.284485 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b168f890-f0a0-41f8-9ea4-a00ef3b9e541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eed51a45b96e45f5b0c7382e00996063'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:40.414033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b168f890-f0a0-41f8-9ea4-a00ef3b9e541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eed51a45b96e45f5b0c7382e00996063'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:40.415266 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b168f890-f0a0-41f8-9ea4-a00ef3b9e541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eed51a45b96e45f5b0c7382e00996063'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:40.416228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b168f890-f0a0-41f8-9ea4-a00ef3b9e541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eed51a45b96e45f5b0c7382e00996063'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:40.444748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b168f890-f0a0-41f8-9ea4-a00ef3b9e541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eed51a45b96e45f5b0c7382e00996063'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:40.486863 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1177/2346] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:39:39 2022] DELETE /identity/v3/users/eed51a45b96e45f5b0c7382e00996063 => generated 0 bytes in 773 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:40.493583 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66a075-6060-4759-91be-75d98f3d4ce5 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:40.494156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66a075-6060-4759-91be-75d98f3d4ce5 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:40.495926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66a075-6060-4759-91be-75d98f3d4ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:40.564424 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-5f66a075-6060-4759-91be-75d98f3d4ce5 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:40.566917 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5f66a075-6060-4759-91be-75d98f3d4ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:40.699521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1170/2347] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:39:40.716073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168f890-f0a0-41f8-9ea4-a00ef3b9e541 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:40.720030 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b168f890-f0a0-41f8-9ea4-a00ef3b9e541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:40.828927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168f890-f0a0-41f8-9ea4-a00ef3b9e541 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:40.838078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a274961-8ac5-4e4f-8d37-5280860a3f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:40.838078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a274961-8ac5-4e4f-8d37-5280860a3f3d service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:40.839411 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a274961-8ac5-4e4f-8d37-5280860a3f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:40.839835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a274961-8ac5-4e4f-8d37-5280860a3f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:40.856420 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a274961-8ac5-4e4f-8d37-5280860a3f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:40.859090 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a274961-8ac5-4e4f-8d37-5280860a3f3d service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:40.882815 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1178/2348] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:39:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:39:41.571105 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66a075-6060-4759-91be-75d98f3d4ce5 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:41.574711 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5f66a075-6060-4759-91be-75d98f3d4ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:41.603167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66a075-6060-4759-91be-75d98f3d4ce5 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:41.613984 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbfb326-6c34-43da-bd28-2db117c07cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:41.617012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfb326-6c34-43da-bd28-2db117c07cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:41.618024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfb326-6c34-43da-bd28-2db117c07cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:41.618839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfb326-6c34-43da-bd28-2db117c07cfd admin admin] PATH_INFO: `/v3/projects/8032bf2514c440fb8b28ca927bb568e3` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:41.625274 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbfb326-6c34-43da-bd28-2db117c07cfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8032bf2514c440fb8b28ca927bb568e3)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:41.633601 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbfb326-6c34-43da-bd28-2db117c07cfd admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:41.883084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:41.916439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5fbfb326-6c34-43da-bd28-2db117c07cfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8032bf2514c440fb8b28ca927bb568e3', 'request_id': 'req-5fbfb326-6c34-43da-bd28-2db117c07cfd'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:41.935419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5fbfb326-6c34-43da-bd28-2db117c07cfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8032bf2514c440fb8b28ca927bb568e3', 'request_id': 'req-5fbfb326-6c34-43da-bd28-2db117c07cfd'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:42.080090 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5fbfb326-6c34-43da-bd28-2db117c07cfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8032bf2514c440fb8b28ca927bb568e3', 'request_id': 'req-5fbfb326-6c34-43da-bd28-2db117c07cfd'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:42.102825 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5fbfb326-6c34-43da-bd28-2db117c07cfd admin admin] The token cache is being invalidate because project 8032bf2514c440fb8b28ca927bb568e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:39:42.103538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5fbfb326-6c34-43da-bd28-2db117c07cfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:42.104655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5fbfb326-6c34-43da-bd28-2db117c07cfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:42.107138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1171/2349] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:39:41 2022] DELETE /identity/v3/projects/8032bf2514c440fb8b28ca927bb568e3 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:42.113700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a274961-8ac5-4e4f-8d37-5280860a3f3d service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:42.121708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:42.136243 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a274961-8ac5-4e4f-8d37-5280860a3f3d service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:42.245141 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849edb9-293e-4b05-a2f7-df128eca001e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:42.246104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849edb9-293e-4b05-a2f7-df128eca001e admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:42.246448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849edb9-293e-4b05-a2f7-df128eca001e admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:42.246783 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849edb9-293e-4b05-a2f7-df128eca001e admin admin] PATH_INFO: `/v3/users/9712c5b5ff4b40a49af079cd2f26b2b4` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:42.249007 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849edb9-293e-4b05-a2f7-df128eca001e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9712c5b5ff4b40a49af079cd2f26b2b4)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:42.251661 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849edb9-293e-4b05-a2f7-df128eca001e admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:42.492028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-7849edb9-293e-4b05-a2f7-df128eca001e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9712c5b5ff4b40a49af079cd2f26b2b4'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:42.587915 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-7849edb9-293e-4b05-a2f7-df128eca001e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9712c5b5ff4b40a49af079cd2f26b2b4'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:42.594015 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-7849edb9-293e-4b05-a2f7-df128eca001e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9712c5b5ff4b40a49af079cd2f26b2b4'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:42.595554 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-7849edb9-293e-4b05-a2f7-df128eca001e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9712c5b5ff4b40a49af079cd2f26b2b4'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:42.609654 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-7849edb9-293e-4b05-a2f7-df128eca001e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9712c5b5ff4b40a49af079cd2f26b2b4'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:42.659614 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1179/2350] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:39:42 2022] DELETE /identity/v3/users/9712c5b5ff4b40a49af079cd2f26b2b4 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:43.968013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbfb326-6c34-43da-bd28-2db117c07cfd admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:43.971022 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5fbfb326-6c34-43da-bd28-2db117c07cfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:43.988665 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbfb326-6c34-43da-bd28-2db117c07cfd admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:44.079254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256c196-0c7c-47e6-a0e2-bd7b62c9df3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:44.080389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256c196-0c7c-47e6-a0e2-bd7b62c9df3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:44.081158 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256c196-0c7c-47e6-a0e2-bd7b62c9df3e admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:44.082163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256c196-0c7c-47e6-a0e2-bd7b62c9df3e admin admin] PATH_INFO: `/v3/projects/90de8d99b02e4d1b9e3f9d967f5a9256` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:44.084698 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256c196-0c7c-47e6-a0e2-bd7b62c9df3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=90de8d99b02e4d1b9e3f9d967f5a9256)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:44.086943 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256c196-0c7c-47e6-a0e2-bd7b62c9df3e admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:44.254593 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5256c196-0c7c-47e6-a0e2-bd7b62c9df3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90de8d99b02e4d1b9e3f9d967f5a9256', 'request_id': 'req-5256c196-0c7c-47e6-a0e2-bd7b62c9df3e'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:44.269152 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5256c196-0c7c-47e6-a0e2-bd7b62c9df3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90de8d99b02e4d1b9e3f9d967f5a9256', 'request_id': 'req-5256c196-0c7c-47e6-a0e2-bd7b62c9df3e'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:44.388062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5256c196-0c7c-47e6-a0e2-bd7b62c9df3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90de8d99b02e4d1b9e3f9d967f5a9256', 'request_id': 'req-5256c196-0c7c-47e6-a0e2-bd7b62c9df3e'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:44.416635 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5256c196-0c7c-47e6-a0e2-bd7b62c9df3e admin admin] The token cache is being invalidate because project 90de8d99b02e4d1b9e3f9d967f5a9256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:39:44.417819 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5256c196-0c7c-47e6-a0e2-bd7b62c9df3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:44.419388 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5256c196-0c7c-47e6-a0e2-bd7b62c9df3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:44.427968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1172/2351] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:39:43 2022] DELETE /identity/v3/projects/90de8d99b02e4d1b9e3f9d967f5a9256 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:44.430495 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b18453-a2d2-4cb9-8477-5464c830edec None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:44.432100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b18453-a2d2-4cb9-8477-5464c830edec None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:44.433357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b18453-a2d2-4cb9-8477-5464c830edec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:44.517298 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-52b18453-a2d2-4cb9-8477-5464c830edec None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:44.520941 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-52b18453-a2d2-4cb9-8477-5464c830edec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:44.653274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1180/2352] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:39:44.675371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7742d-a454-4085-9b4b-3d41b7f06ff2 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:44.675668 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7742d-a454-4085-9b4b-3d41b7f06ff2 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:44.675939 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7742d-a454-4085-9b4b-3d41b7f06ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:44.711050 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-bfa7742d-a454-4085-9b4b-3d41b7f06ff2 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:44.713378 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-bfa7742d-a454-4085-9b4b-3d41b7f06ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:44.729418 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1173/2353] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:39:44.735334 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b18453-a2d2-4cb9-8477-5464c830edec None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:44.747018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b18453-a2d2-4cb9-8477-5464c830edec None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:44.753987 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee18952-12a0-49a3-bc4c-582ef298d4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:44.754682 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee18952-12a0-49a3-bc4c-582ef298d4c9 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:44.754958 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee18952-12a0-49a3-bc4c-582ef298d4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:44.755333 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee18952-12a0-49a3-bc4c-582ef298d4c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:44.756363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee18952-12a0-49a3-bc4c-582ef298d4c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:39:44.757120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee18952-12a0-49a3-bc4c-582ef298d4c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:44.758995 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee18952-12a0-49a3-bc4c-582ef298d4c9 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:44.796453 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1181/2354] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:39:44.797425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7742d-a454-4085-9b4b-3d41b7f06ff2 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:44.809190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7742d-a454-4085-9b4b-3d41b7f06ff2 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:44.815888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932489df-4ed6-4c40-8fd0-d9402e6aac57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:44.816560 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932489df-4ed6-4c40-8fd0-d9402e6aac57 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:44.816859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932489df-4ed6-4c40-8fd0-d9402e6aac57 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:44.817188 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932489df-4ed6-4c40-8fd0-d9402e6aac57 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:44.820628 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932489df-4ed6-4c40-8fd0-d9402e6aac57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:44.823557 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932489df-4ed6-4c40-8fd0-d9402e6aac57 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:45.012884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1174/2355] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:39:44 2022] POST /identity/v3/projects => generated 367 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:39:45.018332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee18952-12a0-49a3-bc4c-582ef298d4c9 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:45.049295 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee18952-12a0-49a3-bc4c-582ef298d4c9 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:45.062312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:45.233517 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49cb48-302b-45de-9754-46ea4ec58522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:45.235139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49cb48-302b-45de-9754-46ea4ec58522 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:45.236157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49cb48-302b-45de-9754-46ea4ec58522 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:45.236922 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49cb48-302b-45de-9754-46ea4ec58522 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:45.244446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49cb48-302b-45de-9754-46ea4ec58522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:45.250491 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49cb48-302b-45de-9754-46ea4ec58522 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:45.384527 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1182/2356] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:39:45 2022] POST /identity/v3/users => generated 284 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:39:45.390237 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932489df-4ed6-4c40-8fd0-d9402e6aac57 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:45.415408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932489df-4ed6-4c40-8fd0-d9402e6aac57 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:45.421931 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664125c-b686-4562-8f7a-76f19eaba02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:45.421931 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664125c-b686-4562-8f7a-76f19eaba02c admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:45.422409 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664125c-b686-4562-8f7a-76f19eaba02c admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:45.422566 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664125c-b686-4562-8f7a-76f19eaba02c admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:45.423188 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664125c-b686-4562-8f7a-76f19eaba02c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:45.425045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664125c-b686-4562-8f7a-76f19eaba02c admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:45.442921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1175/2357] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:39:45.448422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49cb48-302b-45de-9754-46ea4ec58522 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:45.485327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49cb48-302b-45de-9754-46ea4ec58522 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:45.494124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7e400-5cf2-4002-bf1e-1a5d1f2fd35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:45.498020 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7e400-5cf2-4002-bf1e-1a5d1f2fd35a admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:45.500929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7e400-5cf2-4002-bf1e-1a5d1f2fd35a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:45.501894 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7e400-5cf2-4002-bf1e-1a5d1f2fd35a admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:45.505128 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7e400-5cf2-4002-bf1e-1a5d1f2fd35a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:45.511736 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7e400-5cf2-4002-bf1e-1a5d1f2fd35a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:45.552060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1183/2358] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:39:45.557017 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664125c-b686-4562-8f7a-76f19eaba02c admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:45.570757 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664125c-b686-4562-8f7a-76f19eaba02c admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:45.580945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fb455-c04f-4909-b1c9-cf06c66b587e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:45.582283 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fb455-c04f-4909-b1c9-cf06c66b587e admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:45.582719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fb455-c04f-4909-b1c9-cf06c66b587e admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:45.583200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fb455-c04f-4909-b1c9-cf06c66b587e admin admin] PATH_INFO: `/v3/projects/b9bf1d278a5b426f88028b357f4bbb1e/users/6c51d4c9b30f4f39bbe590dad66b4c43/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:45.591452 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:45.618376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fb455-c04f-4909-b1c9-cf06c66b587e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9bf1d278a5b426f88028b357f4bbb1e, user_id=6c51d4c9b30f4f39bbe590dad66b4c43, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:45.622482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fb455-c04f-4909-b1c9-cf06c66b587e admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:45.768969 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1176/2359] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:39:45 2022] PUT /identity/v3/projects/b9bf1d278a5b426f88028b357f4bbb1e/users/6c51d4c9b30f4f39bbe590dad66b4c43/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:45.781699 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3088b53-6985-4ce6-9a6d-d62bc2ffa20c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:45.782661 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3088b53-6985-4ce6-9a6d-d62bc2ffa20c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:45.783374 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3088b53-6985-4ce6-9a6d-d62bc2ffa20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:45.850084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b3088b53-6985-4ce6-9a6d-d62bc2ffa20c None None] MFA Rules not processed for user `6c51d4c9b30f4f39bbe590dad66b4c43`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:45.853284 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b3088b53-6985-4ce6-9a6d-d62bc2ffa20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:46.049552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1184/2360] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:39:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:39:46.054410 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fb455-c04f-4909-b1c9-cf06c66b587e admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:46.299162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fb455-c04f-4909-b1c9-cf06c66b587e admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:46.330137 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc41890-171d-48d1-b56e-f59be10e6ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:46.332726 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc41890-171d-48d1-b56e-f59be10e6ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:46.334047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc41890-171d-48d1-b56e-f59be10e6ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:46.336220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc41890-171d-48d1-b56e-f59be10e6ca6 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:46.345432 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc41890-171d-48d1-b56e-f59be10e6ca6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:46.349455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc41890-171d-48d1-b56e-f59be10e6ca6 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:46.553099 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1177/2361] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:39:46 2022] POST /identity/v3/projects => generated 365 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:39:46.556843 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3088b53-6985-4ce6-9a6d-d62bc2ffa20c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:46.605343 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3088b53-6985-4ce6-9a6d-d62bc2ffa20c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:46.797241 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33091e-33de-4368-8dab-0601641eb35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:46.798361 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33091e-33de-4368-8dab-0601641eb35f admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:46.799295 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33091e-33de-4368-8dab-0601641eb35f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:46.799799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33091e-33de-4368-8dab-0601641eb35f admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:46.801198 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33091e-33de-4368-8dab-0601641eb35f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:46.804497 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33091e-33de-4368-8dab-0601641eb35f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:46.991177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1185/2362] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:39:46 2022] POST /identity/v3/users => generated 289 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:39:46.997460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc41890-171d-48d1-b56e-f59be10e6ca6 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:47.026213 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc41890-171d-48d1-b56e-f59be10e6ca6 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:47.035273 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfafbf8-1175-4a6d-8d7e-73df1cd5a073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:47.036749 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfafbf8-1175-4a6d-8d7e-73df1cd5a073 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:47.037303 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfafbf8-1175-4a6d-8d7e-73df1cd5a073 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:47.037962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfafbf8-1175-4a6d-8d7e-73df1cd5a073 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:47.043076 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfafbf8-1175-4a6d-8d7e-73df1cd5a073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:47.047022 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfafbf8-1175-4a6d-8d7e-73df1cd5a073 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:47.078837 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1178/2363] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:39:47.082214 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33091e-33de-4368-8dab-0601641eb35f admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:47.108457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33091e-33de-4368-8dab-0601641eb35f admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:47.122735 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ade3d-623b-47cf-a874-a2a5cba2e5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:47.125140 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ade3d-623b-47cf-a874-a2a5cba2e5e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:47.125687 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ade3d-623b-47cf-a874-a2a5cba2e5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:47.126089 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ade3d-623b-47cf-a874-a2a5cba2e5e8 admin admin] PATH_INFO: `/v3/projects/913beb09449b4a6e91f8586bd153964c/users/e0bc635d98254a4d9964e2a3f73765e5/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:47.130875 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:47.166362 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ade3d-623b-47cf-a874-a2a5cba2e5e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=913beb09449b4a6e91f8586bd153964c, user_id=e0bc635d98254a4d9964e2a3f73765e5, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:47.178799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ade3d-623b-47cf-a874-a2a5cba2e5e8 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:47.383715 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1186/2364] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:39:47 2022] PUT /identity/v3/projects/913beb09449b4a6e91f8586bd153964c/users/e0bc635d98254a4d9964e2a3f73765e5/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:47.393239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfafbf8-1175-4a6d-8d7e-73df1cd5a073 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:47.419348 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfafbf8-1175-4a6d-8d7e-73df1cd5a073 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:47.591328 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b5e86-1475-4eaa-830e-b7ab414bd553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:47.592730 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b5e86-1475-4eaa-830e-b7ab414bd553 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:47.593226 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b5e86-1475-4eaa-830e-b7ab414bd553 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:47.593717 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b5e86-1475-4eaa-830e-b7ab414bd553 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:47.595012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b5e86-1475-4eaa-830e-b7ab414bd553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:47.598286 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b5e86-1475-4eaa-830e-b7ab414bd553 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:47.636642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1179/2365] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:39:47.641911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ade3d-623b-47cf-a874-a2a5cba2e5e8 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:47.659973 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ade3d-623b-47cf-a874-a2a5cba2e5e8 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:47.671248 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e8a81-5437-430f-bb46-4f77a8d1a5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:47.672522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e8a81-5437-430f-bb46-4f77a8d1a5e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:47.672895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e8a81-5437-430f-bb46-4f77a8d1a5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:47.673339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e8a81-5437-430f-bb46-4f77a8d1a5e0 admin admin] PATH_INFO: `/v3/projects/913beb09449b4a6e91f8586bd153964c/users/e0bc635d98254a4d9964e2a3f73765e5/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:47.677352 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e8a81-5437-430f-bb46-4f77a8d1a5e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=913beb09449b4a6e91f8586bd153964c, user_id=e0bc635d98254a4d9964e2a3f73765e5, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:47.682529 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e8a81-5437-430f-bb46-4f77a8d1a5e0 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:47.699736 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1187/2366] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:39:47 2022] PUT /identity/v3/projects/913beb09449b4a6e91f8586bd153964c/users/e0bc635d98254a4d9964e2a3f73765e5/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:47.711961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afe3cb-4d20-441a-82d5-20cf319aea12 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:47.711961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afe3cb-4d20-441a-82d5-20cf319aea12 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:47.712405 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53afe3cb-4d20-441a-82d5-20cf319aea12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:47.755036 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-53afe3cb-4d20-441a-82d5-20cf319aea12 None None] MFA Rules not processed for user `e0bc635d98254a4d9964e2a3f73765e5`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:47.755036 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-53afe3cb-4d20-441a-82d5-20cf319aea12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:47.850764 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:47.914517 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1180/2367] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:39:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:39:47.928375 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e8a81-5437-430f-bb46-4f77a8d1a5e0 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:47.930622 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-3e8e8a81-5437-430f-bb46-4f77a8d1a5e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:48.021309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e8a81-5437-430f-bb46-4f77a8d1a5e0 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:48.026032 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a39faf-a18c-43c6-9c31-1c7b4758c1ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:48.026810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a39faf-a18c-43c6-9c31-1c7b4758c1ad service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:48.027050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a39faf-a18c-43c6-9c31-1c7b4758c1ad service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:48.027270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a39faf-a18c-43c6-9c31-1c7b4758c1ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:48.038492 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a39faf-a18c-43c6-9c31-1c7b4758c1ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:48.040126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a39faf-a18c-43c6-9c31-1c7b4758c1ad service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:48.054649 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1188/2368] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:39:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:39:48.584459 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53afe3cb-4d20-441a-82d5-20cf319aea12 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:48.610820 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53afe3cb-4d20-441a-82d5-20cf319aea12 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:48.631447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdab773-ad80-42cc-8416-2d5a4af07f72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:48.632558 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdab773-ad80-42cc-8416-2d5a4af07f72 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:48.633062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdab773-ad80-42cc-8416-2d5a4af07f72 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:48.633555 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdab773-ad80-42cc-8416-2d5a4af07f72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:48.797995 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdab773-ad80-42cc-8416-2d5a4af07f72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:48.801063 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdab773-ad80-42cc-8416-2d5a4af07f72 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:48.831160 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1181/2369] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:39:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:39:58.049627 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=38,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:58.243932 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a39faf-a18c-43c6-9c31-1c7b4758c1ad service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:58.348462 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a39faf-a18c-43c6-9c31-1c7b4758c1ad service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:58.354688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d911a12-bbb5-4990-ab8e-363769aa5b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:58.355811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d911a12-bbb5-4990-ab8e-363769aa5b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:58.356149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d911a12-bbb5-4990-ab8e-363769aa5b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:58.356584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d911a12-bbb5-4990-ab8e-363769aa5b3a admin admin] PATH_INFO: `/v3/users/6c51d4c9b30f4f39bbe590dad66b4c43` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:58.359252 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d911a12-bbb5-4990-ab8e-363769aa5b3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c51d4c9b30f4f39bbe590dad66b4c43)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:58.362741 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d911a12-bbb5-4990-ab8e-363769aa5b3a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:58.697602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-6d911a12-bbb5-4990-ab8e-363769aa5b3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c51d4c9b30f4f39bbe590dad66b4c43'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:58.809975 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-6d911a12-bbb5-4990-ab8e-363769aa5b3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c51d4c9b30f4f39bbe590dad66b4c43'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:58.811621 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-6d911a12-bbb5-4990-ab8e-363769aa5b3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c51d4c9b30f4f39bbe590dad66b4c43'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:58.818042 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-6d911a12-bbb5-4990-ab8e-363769aa5b3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c51d4c9b30f4f39bbe590dad66b4c43'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:58.818283 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=37 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:58.824117 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dbf67-da15-4969-92b0-3a3e24f69a44 None None] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:58.824610 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dbf67-da15-4969-92b0-3a3e24f69a44 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:58.825058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dbf67-da15-4969-92b0-3a3e24f69a44 None None] PATH_INFO: `/` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:58.827675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1182/2370] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 7 13:39:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 13:39:58.833425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-6d911a12-bbb5-4990-ab8e-363769aa5b3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c51d4c9b30f4f39bbe590dad66b4c43'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:39:58.836322 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157232b-287c-4b39-b2fd-48eb7c72ac96 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:58.852800 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157232b-287c-4b39-b2fd-48eb7c72ac96 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:58.852800 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157232b-287c-4b39-b2fd-48eb7c72ac96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:58.869712 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-1157232b-287c-4b39-b2fd-48eb7c72ac96 None None] MFA Rules not processed for user `97c363046db444cbb3d4346066f9ad42`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:58.871212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-1157232b-287c-4b39-b2fd-48eb7c72ac96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:58.877788 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1189/2371] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:39:58 2022] DELETE /identity/v3/users/6c51d4c9b30f4f39bbe590dad66b4c43 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:39:58.892744 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d34d09-5a1d-4f5c-a3c6-fc230bd61cd6 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:58.892744 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d34d09-5a1d-4f5c-a3c6-fc230bd61cd6 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:58.892744 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d34d09-5a1d-4f5c-a3c6-fc230bd61cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:58.915800 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-e3d34d09-5a1d-4f5c-a3c6-fc230bd61cd6 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:58.917042 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e3d34d09-5a1d-4f5c-a3c6-fc230bd61cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:58.988225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1183/2372] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 7 13:39:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:39:59.015248 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157232b-287c-4b39-b2fd-48eb7c72ac96 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:59.017646 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-1157232b-287c-4b39-b2fd-48eb7c72ac96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:59.033053 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1190/2373] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:39:59.047255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d34d09-5a1d-4f5c-a3c6-fc230bd61cd6 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:59.049558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e3d34d09-5a1d-4f5c-a3c6-fc230bd61cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:59.080049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:59.117403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157232b-287c-4b39-b2fd-48eb7c72ac96 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:59.140714 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d34d09-5a1d-4f5c-a3c6-fc230bd61cd6 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:59.146286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e207d1-6fbf-416c-8551-9672cb75e182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:59.147706 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e207d1-6fbf-416c-8551-9672cb75e182 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:59.148111 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e207d1-6fbf-416c-8551-9672cb75e182 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:59.148584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e207d1-6fbf-416c-8551-9672cb75e182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:59.166831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e207d1-6fbf-416c-8551-9672cb75e182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:59.168710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e207d1-6fbf-416c-8551-9672cb75e182 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:59.182148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1191/2374] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:39:59.195848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c093b1-dcb3-43b1-b02d-ece1f0481e18 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1a48805d95640489d7ce3333dd687b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:59.196667 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c093b1-dcb3-43b1-b02d-ece1f0481e18 service glance] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:59.196925 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c093b1-dcb3-43b1-b02d-ece1f0481e18 service glance] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:59.197214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c093b1-dcb3-43b1-b02d-ece1f0481e18 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:59.292417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:39:59.314297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c093b1-dcb3-43b1-b02d-ece1f0481e18 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:59.316771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c093b1-dcb3-43b1-b02d-ece1f0481e18 service glance] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:59.332693 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1184/2375] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:39:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:39:59.652528 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03588c67-fd1b-489f-b667-d71f83a38f13 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:59.653024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03588c67-fd1b-489f-b667-d71f83a38f13 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:59.653504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03588c67-fd1b-489f-b667-d71f83a38f13 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:59.656431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1192/2376] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 7 13:39:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:39:59.662480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619e431-9519-4245-9dd8-a31b74f4c7ca None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:59.663279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619e431-9519-4245-9dd8-a31b74f4c7ca None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:59.663551 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619e431-9519-4245-9dd8-a31b74f4c7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:59.695200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c619e431-9519-4245-9dd8-a31b74f4c7ca None None] MFA Rules not processed for user `5acdbdb6f23b4f10bcbab0387ea67ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:59.697355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c619e431-9519-4245-9dd8-a31b74f4c7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:59.711350 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1185/2377] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 7 13:39:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:39:59.726245 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03588c67-fd1b-489f-b667-d71f83a38f13 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:59.729141 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-03588c67-fd1b-489f-b667-d71f83a38f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:59.754627 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03588c67-fd1b-489f-b667-d71f83a38f13 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:59.761062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e110015-54c7-4989-b228-f78aae9d922a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:59.762086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e110015-54c7-4989-b228-f78aae9d922a service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:59.762567 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e110015-54c7-4989-b228-f78aae9d922a service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:59.763094 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e110015-54c7-4989-b228-f78aae9d922a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:59.778351 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e110015-54c7-4989-b228-f78aae9d922a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:59.781690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e110015-54c7-4989-b228-f78aae9d922a service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:59.798789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1193/2378] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:39:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:39:59.887391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619e431-9519-4245-9dd8-a31b74f4c7ca None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:39:59.887391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c619e431-9519-4245-9dd8-a31b74f4c7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:59.908180 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619e431-9519-4245-9dd8-a31b74f4c7ca None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:39:59.922007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:39:59.923522 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:59.924335 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:59.924816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617 admin admin] PATH_INFO: `/v3/projects/b9bf1d278a5b426f88028b357f4bbb1e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:59.928408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9bf1d278a5b426f88028b357f4bbb1e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:39:59.932805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:39:59.940605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cee93f-b8e3-41f7-bd6b-3ff50068bb82 None None] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:59.941095 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cee93f-b8e3-41f7-bd6b-3ff50068bb82 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:59.941613 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cee93f-b8e3-41f7-bd6b-3ff50068bb82 None None] PATH_INFO: `/` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:59.946012 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1194/2379] 10.222.0.62 () {54 vars in 921 bytes} [Wed Sep 7 13:39:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 13:39:59.954340 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41031c64-1d6b-40fd-aea5-03a9773d6ba9 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:39:59.954531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41031c64-1d6b-40fd-aea5-03a9773d6ba9 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:39:59.954882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41031c64-1d6b-40fd-aea5-03a9773d6ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:39:59.985305 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-41031c64-1d6b-40fd-aea5-03a9773d6ba9 None None] MFA Rules not processed for user `97c363046db444cbb3d4346066f9ad42`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:39:59.986702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-41031c64-1d6b-40fd-aea5-03a9773d6ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:39:59.999628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1195/2380] 10.222.0.62 () {58 vars in 1018 bytes} [Wed Sep 7 13:39:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:40:00.014535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41031c64-1d6b-40fd-aea5-03a9773d6ba9 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:00.028294 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41031c64-1d6b-40fd-aea5-03a9773d6ba9 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:00.036031 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4fe21-0d03-448e-9338-161a7475e15e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:00.037379 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4fe21-0d03-448e-9338-161a7475e15e service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:00.037937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4fe21-0d03-448e-9338-161a7475e15e service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:00.038529 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4fe21-0d03-448e-9338-161a7475e15e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:00.058744 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4fe21-0d03-448e-9338-161a7475e15e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:00.062496 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4fe21-0d03-448e-9338-161a7475e15e service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:00.080980 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1196/2381] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:40:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:40:00.126373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9bf1d278a5b426f88028b357f4bbb1e', 'request_id': 'req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:00.141606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9bf1d278a5b426f88028b357f4bbb1e', 'request_id': 'req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:00.277224 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9bf1d278a5b426f88028b357f4bbb1e', 'request_id': 'req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:00.301005 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617 admin admin] The token cache is being invalidate because project b9bf1d278a5b426f88028b357f4bbb1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:40:00.301514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:00.302780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:00.305450 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1186/2382] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:39:59 2022] DELETE /identity/v3/projects/b9bf1d278a5b426f88028b357f4bbb1e => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:00.310063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4fe21-0d03-448e-9338-161a7475e15e service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:00.324472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4fe21-0d03-448e-9338-161a7475e15e service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:00.407891 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6632316-7212-4b3a-a6bc-fc8c9595c967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:00.409016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6632316-7212-4b3a-a6bc-fc8c9595c967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:00.409301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6632316-7212-4b3a-a6bc-fc8c9595c967 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:00.409576 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6632316-7212-4b3a-a6bc-fc8c9595c967 admin admin] PATH_INFO: `/v3/users/e0bc635d98254a4d9964e2a3f73765e5` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:00.411150 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6632316-7212-4b3a-a6bc-fc8c9595c967 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0bc635d98254a4d9964e2a3f73765e5)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:00.413620 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6632316-7212-4b3a-a6bc-fc8c9595c967 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:00.694001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-d6632316-7212-4b3a-a6bc-fc8c9595c967 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0bc635d98254a4d9964e2a3f73765e5'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:00.849725 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-d6632316-7212-4b3a-a6bc-fc8c9595c967 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0bc635d98254a4d9964e2a3f73765e5'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:00.850558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-d6632316-7212-4b3a-a6bc-fc8c9595c967 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0bc635d98254a4d9964e2a3f73765e5'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:00.852263 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-d6632316-7212-4b3a-a6bc-fc8c9595c967 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0bc635d98254a4d9964e2a3f73765e5'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:00.863769 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-d6632316-7212-4b3a-a6bc-fc8c9595c967 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0bc635d98254a4d9964e2a3f73765e5'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:00.890607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1197/2383] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:40:00 2022] DELETE /identity/v3/users/e0bc635d98254a4d9964e2a3f73765e5 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:01.578561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:01.580921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:01.629461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3ab99-e7b6-4d8a-a2ac-51c0ebd26617 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:01.721246 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b1d69-d0d7-482e-beff-de7668ff8c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:01.722391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b1d69-d0d7-482e-beff-de7668ff8c84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:01.722896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b1d69-d0d7-482e-beff-de7668ff8c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:01.723452 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b1d69-d0d7-482e-beff-de7668ff8c84 admin admin] PATH_INFO: `/v3/projects/913beb09449b4a6e91f8586bd153964c` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:01.725757 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b1d69-d0d7-482e-beff-de7668ff8c84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=913beb09449b4a6e91f8586bd153964c)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:01.729780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b1d69-d0d7-482e-beff-de7668ff8c84 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:01.937341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5f0b1d69-d0d7-482e-beff-de7668ff8c84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '913beb09449b4a6e91f8586bd153964c', 'request_id': 'req-5f0b1d69-d0d7-482e-beff-de7668ff8c84'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:01.946050 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5f0b1d69-d0d7-482e-beff-de7668ff8c84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '913beb09449b4a6e91f8586bd153964c', 'request_id': 'req-5f0b1d69-d0d7-482e-beff-de7668ff8c84'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.071143 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5f0b1d69-d0d7-482e-beff-de7668ff8c84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '913beb09449b4a6e91f8586bd153964c', 'request_id': 'req-5f0b1d69-d0d7-482e-beff-de7668ff8c84'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.079023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:02.086727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5f0b1d69-d0d7-482e-beff-de7668ff8c84 admin admin] The token cache is being invalidate because project 913beb09449b4a6e91f8586bd153964c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:40:02.087063 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5f0b1d69-d0d7-482e-beff-de7668ff8c84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.088272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-5f0b1d69-d0d7-482e-beff-de7668ff8c84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:02.089997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1187/2384] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:40:01 2022] DELETE /identity/v3/projects/913beb09449b4a6e91f8586bd153964c => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:02.096157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1aac2-de75-4558-8322-a760baba26de None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:02.096980 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1aac2-de75-4558-8322-a760baba26de None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:02.096980 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1aac2-de75-4558-8322-a760baba26de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:02.122867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-5bd1aac2-de75-4558-8322-a760baba26de None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:02.125355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-5bd1aac2-de75-4558-8322-a760baba26de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:02.133160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:02.244033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1198/2385] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:40:02.256864 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a2ff3-b1ee-4fcf-9c93-43b4cb0a3b41 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:02.259882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a2ff3-b1ee-4fcf-9c93-43b4cb0a3b41 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:02.259882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a2ff3-b1ee-4fcf-9c93-43b4cb0a3b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:02.289192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-009a2ff3-b1ee-4fcf-9c93-43b4cb0a3b41 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:02.292669 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-009a2ff3-b1ee-4fcf-9c93-43b4cb0a3b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:02.300777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1188/2386] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:02.305494 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1aac2-de75-4558-8322-a760baba26de None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:02.316522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1aac2-de75-4558-8322-a760baba26de None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:02.321679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c2d3f-25e0-4db6-b9c7-aeb5833486c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:02.322330 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c2d3f-25e0-4db6-b9c7-aeb5833486c0 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:02.322622 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c2d3f-25e0-4db6-b9c7-aeb5833486c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:02.322896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c2d3f-25e0-4db6-b9c7-aeb5833486c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:02.323655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c2d3f-25e0-4db6-b9c7-aeb5833486c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:40:02.323961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c2d3f-25e0-4db6-b9c7-aeb5833486c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:02.325673 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c2d3f-25e0-4db6-b9c7-aeb5833486c0 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:02.340520 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1199/2387] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:40:02.346493 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a2ff3-b1ee-4fcf-9c93-43b4cb0a3b41 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:02.357065 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a2ff3-b1ee-4fcf-9c93-43b4cb0a3b41 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:02.362586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136badd3-8273-4d15-acb5-b0045504e939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:02.363962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136badd3-8273-4d15-acb5-b0045504e939 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:02.364414 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136badd3-8273-4d15-acb5-b0045504e939 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:02.365325 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136badd3-8273-4d15-acb5-b0045504e939 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:02.366216 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136badd3-8273-4d15-acb5-b0045504e939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:02.368527 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136badd3-8273-4d15-acb5-b0045504e939 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:02.528350 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1189/2388] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:40:02 2022] POST /identity/v3/projects => generated 375 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:40:02.534638 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c2d3f-25e0-4db6-b9c7-aeb5833486c0 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:02.548378 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c2d3f-25e0-4db6-b9c7-aeb5833486c0 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:02.664232 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79365622-a291-4463-9adf-5ab43c48a23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:02.665141 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79365622-a291-4463-9adf-5ab43c48a23b admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:02.665542 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79365622-a291-4463-9adf-5ab43c48a23b admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:02.666003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79365622-a291-4463-9adf-5ab43c48a23b admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:02.668847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79365622-a291-4463-9adf-5ab43c48a23b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:02.671750 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79365622-a291-4463-9adf-5ab43c48a23b admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:02.825031 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1200/2389] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:40:02 2022] POST /identity/v3/users => generated 288 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:40:02.831253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136badd3-8273-4d15-acb5-b0045504e939 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:02.846742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136badd3-8273-4d15-acb5-b0045504e939 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:02.853658 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca0d7c-ae93-480d-aae4-e32dafb4f881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:02.855732 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca0d7c-ae93-480d-aae4-e32dafb4f881 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:02.855732 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca0d7c-ae93-480d-aae4-e32dafb4f881 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:02.855732 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca0d7c-ae93-480d-aae4-e32dafb4f881 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:02.857091 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca0d7c-ae93-480d-aae4-e32dafb4f881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:02.859595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca0d7c-ae93-480d-aae4-e32dafb4f881 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:02.880959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1190/2390] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:40:02.883149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79365622-a291-4463-9adf-5ab43c48a23b admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:02.907485 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79365622-a291-4463-9adf-5ab43c48a23b admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:02.927125 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cf97d-2998-42ab-96d6-e49d30090a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:02.928532 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cf97d-2998-42ab-96d6-e49d30090a80 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:02.929255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cf97d-2998-42ab-96d6-e49d30090a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:02.930049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cf97d-2998-42ab-96d6-e49d30090a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:02.931811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cf97d-2998-42ab-96d6-e49d30090a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:02.936096 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cf97d-2998-42ab-96d6-e49d30090a80 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:02.962677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1201/2391] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:40:02.968469 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca0d7c-ae93-480d-aae4-e32dafb4f881 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:02.982574 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca0d7c-ae93-480d-aae4-e32dafb4f881 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:02.989408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f761456-b44e-4234-9c19-9a40fd05dbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:02.990328 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f761456-b44e-4234-9c19-9a40fd05dbb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:02.990722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f761456-b44e-4234-9c19-9a40fd05dbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:02.991266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f761456-b44e-4234-9c19-9a40fd05dbb9 admin admin] PATH_INFO: `/v3/projects/cf4569d8c5784fadb8f0cea716a5d5b3/users/79f64f03029647d0ba88d98500077788/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:03.011263 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f761456-b44e-4234-9c19-9a40fd05dbb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf4569d8c5784fadb8f0cea716a5d5b3, user_id=79f64f03029647d0ba88d98500077788, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:03.015197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f761456-b44e-4234-9c19-9a40fd05dbb9 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:03.158441 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1191/2392] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:40:02 2022] PUT /identity/v3/projects/cf4569d8c5784fadb8f0cea716a5d5b3/users/79f64f03029647d0ba88d98500077788/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:40:03.174348 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fe624-e8df-4188-92bc-d73a850c88b4 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:03.175346 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fe624-e8df-4188-92bc-d73a850c88b4 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:03.175346 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fe624-e8df-4188-92bc-d73a850c88b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:03.213579 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-8c7fe624-e8df-4188-92bc-d73a850c88b4 None None] MFA Rules not processed for user `79f64f03029647d0ba88d98500077788`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:03.215221 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8c7fe624-e8df-4188-92bc-d73a850c88b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:03.315564 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1202/2393] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:03.322852 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f761456-b44e-4234-9c19-9a40fd05dbb9 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:03.412178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f761456-b44e-4234-9c19-9a40fd05dbb9 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:03.421426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ebc35-89cd-4336-b7e1-80f6005bcfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:03.422592 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ebc35-89cd-4336-b7e1-80f6005bcfa5 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:03.423124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ebc35-89cd-4336-b7e1-80f6005bcfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:03.423601 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ebc35-89cd-4336-b7e1-80f6005bcfa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:03.430643 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ebc35-89cd-4336-b7e1-80f6005bcfa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:03.430643 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ebc35-89cd-4336-b7e1-80f6005bcfa5 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:03.573007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1192/2394] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:40:03 2022] POST /identity/v3/projects => generated 375 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:40:03.580100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7fe624-e8df-4188-92bc-d73a850c88b4 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:03.592301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:03.601663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7fe624-e8df-4188-92bc-d73a850c88b4 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:03.681444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd828de-688c-464e-9fc0-025d2241c022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:03.682298 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd828de-688c-464e-9fc0-025d2241c022 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:03.682779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd828de-688c-464e-9fc0-025d2241c022 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:03.683163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd828de-688c-464e-9fc0-025d2241c022 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:03.688595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd828de-688c-464e-9fc0-025d2241c022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:03.688595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd828de-688c-464e-9fc0-025d2241c022 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:03.817972 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1203/2395] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:40:03 2022] POST /identity/v3/users => generated 288 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:40:03.823983 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ebc35-89cd-4336-b7e1-80f6005bcfa5 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:03.849245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ebc35-89cd-4336-b7e1-80f6005bcfa5 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:03.858842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f108c254-166d-4eed-8286-455f4e0663b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:03.859981 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108c254-166d-4eed-8286-455f4e0663b9 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:03.860437 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108c254-166d-4eed-8286-455f4e0663b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:03.860935 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108c254-166d-4eed-8286-455f4e0663b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:03.862040 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f108c254-166d-4eed-8286-455f4e0663b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:03.864941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f108c254-166d-4eed-8286-455f4e0663b9 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:03.889669 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1193/2396] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:40:03.895472 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd828de-688c-464e-9fc0-025d2241c022 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:03.906322 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd828de-688c-464e-9fc0-025d2241c022 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:03.913557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa5dc2-2ffb-4d27-860e-4a4cf921c75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:03.914325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa5dc2-2ffb-4d27-860e-4a4cf921c75e admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:03.914723 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa5dc2-2ffb-4d27-860e-4a4cf921c75e admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:03.915146 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa5dc2-2ffb-4d27-860e-4a4cf921c75e admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:03.915944 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa5dc2-2ffb-4d27-860e-4a4cf921c75e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:03.917921 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa5dc2-2ffb-4d27-860e-4a4cf921c75e admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:03.930958 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1204/2397] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:40:03.937055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f108c254-166d-4eed-8286-455f4e0663b9 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:03.945408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73205ad0-4869-4d19-a208-88c15658aafc None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:03.945837 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73205ad0-4869-4d19-a208-88c15658aafc None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:03.946159 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73205ad0-4869-4d19-a208-88c15658aafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:03.954332 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f108c254-166d-4eed-8286-455f4e0663b9 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:03.960684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4fe4d-8416-4df1-9736-fb3af3257386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:03.961395 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4fe4d-8416-4df1-9736-fb3af3257386 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:03.961710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4fe4d-8416-4df1-9736-fb3af3257386 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:03.961948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4fe4d-8416-4df1-9736-fb3af3257386 admin admin] PATH_INFO: `/v3/projects/3e0195841981464c9eebdcc54dcb7ecd/users/1b9774ebcccb46449b62315cde31c20f/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:03.974005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-73205ad0-4869-4d19-a208-88c15658aafc None None] MFA Rules not processed for user `8121ba4cf2cf44d9a5a55975ea85928c`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:03.975329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-73205ad0-4869-4d19-a208-88c15658aafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:03.985982 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4fe4d-8416-4df1-9736-fb3af3257386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e0195841981464c9eebdcc54dcb7ecd, user_id=1b9774ebcccb46449b62315cde31c20f, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:03.991316 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4fe4d-8416-4df1-9736-fb3af3257386 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:04.086322 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1205/2398] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 7 13:40:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:40:04.102150 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73205ad0-4869-4d19-a208-88c15658aafc None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:04.107609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-73205ad0-4869-4d19-a208-88c15658aafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:04.107791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1194/2399] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:40:03 2022] PUT /identity/v3/projects/3e0195841981464c9eebdcc54dcb7ecd/users/1b9774ebcccb46449b62315cde31c20f/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:04.127024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadf680-a797-49cc-9c5f-5104ea26f22e None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:04.127415 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadf680-a797-49cc-9c5f-5104ea26f22e None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:04.127922 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadf680-a797-49cc-9c5f-5104ea26f22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:04.177616 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-2aadf680-a797-49cc-9c5f-5104ea26f22e None None] MFA Rules not processed for user `1b9774ebcccb46449b62315cde31c20f`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:04.179859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2aadf680-a797-49cc-9c5f-5104ea26f22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:04.231408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:04.244423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73205ad0-4869-4d19-a208-88c15658aafc None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:04.277553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1195/2400] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:04.282552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadf680-a797-49cc-9c5f-5104ea26f22e None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:04.303987 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:04.345218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fb6a7-0bfe-4009-9b0e-e48af4c17b48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cedc7091d2f46b5be68818d26399e9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:04.345905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fb6a7-0bfe-4009-9b0e-e48af4c17b48 service swift] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:04.346280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fb6a7-0bfe-4009-9b0e-e48af4c17b48 service swift] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:04.346794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fb6a7-0bfe-4009-9b0e-e48af4c17b48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:04.365976 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadf680-a797-49cc-9c5f-5104ea26f22e None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:04.372181 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2987563-3f43-423f-812e-8c7643f1c503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:04.373640 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2987563-3f43-423f-812e-8c7643f1c503 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:04.374094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2987563-3f43-423f-812e-8c7643f1c503 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:04.374432 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2987563-3f43-423f-812e-8c7643f1c503 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:04.375338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2987563-3f43-423f-812e-8c7643f1c503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:04.377780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2987563-3f43-423f-812e-8c7643f1c503 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:04.436464 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fb6a7-0bfe-4009-9b0e-e48af4c17b48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:04.438232 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fb6a7-0bfe-4009-9b0e-e48af4c17b48 service swift] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:04.447956 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1206/2401] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 7 13:40:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 13:40:04.499719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1196/2402] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:40:04 2022] POST /identity/v3/projects => generated 375 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:40:04.504924 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fb6a7-0bfe-4009-9b0e-e48af4c17b48 service swift] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:04.524656 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fb6a7-0bfe-4009-9b0e-e48af4c17b48 service swift] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:04.604593 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453d615-4ea8-42e6-ad06-1093ae2e599a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:04.605037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453d615-4ea8-42e6-ad06-1093ae2e599a admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:04.605344 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453d615-4ea8-42e6-ad06-1093ae2e599a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:04.605720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453d615-4ea8-42e6-ad06-1093ae2e599a admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:04.606407 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453d615-4ea8-42e6-ad06-1093ae2e599a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:04.608671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453d615-4ea8-42e6-ad06-1093ae2e599a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:04.785286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1207/2403] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:40:04 2022] POST /identity/v3/users => generated 294 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:40:04.792960 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2987563-3f43-423f-812e-8c7643f1c503 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:04.841228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2987563-3f43-423f-812e-8c7643f1c503 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:04.857358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e1bf2-cdb2-437c-b31a-bcf2fd74e96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:04.861601 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e1bf2-cdb2-437c-b31a-bcf2fd74e96f admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:04.863496 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e1bf2-cdb2-437c-b31a-bcf2fd74e96f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:04.864910 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e1bf2-cdb2-437c-b31a-bcf2fd74e96f admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:04.866893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e1bf2-cdb2-437c-b31a-bcf2fd74e96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:04.889466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e1bf2-cdb2-437c-b31a-bcf2fd74e96f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:04.922362 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1197/2404] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:40:04.929072 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453d615-4ea8-42e6-ad06-1093ae2e599a admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:04.949125 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453d615-4ea8-42e6-ad06-1093ae2e599a admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:04.956231 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519c181-2eaa-4448-8132-1308fd06f357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:04.957057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519c181-2eaa-4448-8132-1308fd06f357 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:04.957443 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519c181-2eaa-4448-8132-1308fd06f357 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:04.957821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519c181-2eaa-4448-8132-1308fd06f357 admin admin] PATH_INFO: `/v3/projects/ad60527e642a41e6a2e6b0b07017b63d/users/03aad8b7cef448a888b48729dbe6cdc9/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:04.981666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519c181-2eaa-4448-8132-1308fd06f357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad60527e642a41e6a2e6b0b07017b63d, user_id=03aad8b7cef448a888b48729dbe6cdc9, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:04.986441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519c181-2eaa-4448-8132-1308fd06f357 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:05.100892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1208/2405] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:40:04 2022] PUT /identity/v3/projects/ad60527e642a41e6a2e6b0b07017b63d/users/03aad8b7cef448a888b48729dbe6cdc9/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:05.108545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e1bf2-cdb2-437c-b31a-bcf2fd74e96f admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:05.121669 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e1bf2-cdb2-437c-b31a-bcf2fd74e96f admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:05.266357 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d4fc1-e05e-4e51-9bb8-3cdfe4ff276f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:05.267432 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d4fc1-e05e-4e51-9bb8-3cdfe4ff276f admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:05.267732 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d4fc1-e05e-4e51-9bb8-3cdfe4ff276f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:05.268009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d4fc1-e05e-4e51-9bb8-3cdfe4ff276f admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:05.268875 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d4fc1-e05e-4e51-9bb8-3cdfe4ff276f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:05.270574 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d4fc1-e05e-4e51-9bb8-3cdfe4ff276f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:05.303609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1198/2406] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:40:05.305046 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519c181-2eaa-4448-8132-1308fd06f357 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:05.331330 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519c181-2eaa-4448-8132-1308fd06f357 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:05.336853 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38c8d6-d1be-4afe-bd08-aa9c6bef152f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:05.349266 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38c8d6-d1be-4afe-bd08-aa9c6bef152f admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:05.350466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38c8d6-d1be-4afe-bd08-aa9c6bef152f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:05.350864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38c8d6-d1be-4afe-bd08-aa9c6bef152f admin admin] PATH_INFO: `/v3/projects/ad60527e642a41e6a2e6b0b07017b63d/users/03aad8b7cef448a888b48729dbe6cdc9/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:05.358079 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38c8d6-d1be-4afe-bd08-aa9c6bef152f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad60527e642a41e6a2e6b0b07017b63d, user_id=03aad8b7cef448a888b48729dbe6cdc9, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:05.361674 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38c8d6-d1be-4afe-bd08-aa9c6bef152f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:05.384254 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1209/2407] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:40:05 2022] PUT /identity/v3/projects/ad60527e642a41e6a2e6b0b07017b63d/users/03aad8b7cef448a888b48729dbe6cdc9/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:05.403224 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53efac8-e7c9-40da-9fcc-7956c2506a99 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:05.403723 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53efac8-e7c9-40da-9fcc-7956c2506a99 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:05.404306 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53efac8-e7c9-40da-9fcc-7956c2506a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:05.508598 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-a53efac8-e7c9-40da-9fcc-7956c2506a99 None None] MFA Rules not processed for user `03aad8b7cef448a888b48729dbe6cdc9`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:05.511463 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a53efac8-e7c9-40da-9fcc-7956c2506a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:05.601058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:05.717732 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1199/2408] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:05.734743 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38c8d6-d1be-4afe-bd08-aa9c6bef152f admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:05.737625 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ec38c8d6-d1be-4afe-bd08-aa9c6bef152f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:05.791039 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:05.862408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38c8d6-d1be-4afe-bd08-aa9c6bef152f admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:05.868034 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fa319-0a74-4953-8f4c-f79e90a43105 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:05.868643 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fa319-0a74-4953-8f4c-f79e90a43105 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:05.868851 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fa319-0a74-4953-8f4c-f79e90a43105 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:05.869129 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fa319-0a74-4953-8f4c-f79e90a43105 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:06.005253 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fa319-0a74-4953-8f4c-f79e90a43105 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:06.008243 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fa319-0a74-4953-8f4c-f79e90a43105 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:06.024084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1210/2409] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:40:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:40:09.930369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53efac8-e7c9-40da-9fcc-7956c2506a99 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:09.932978 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a53efac8-e7c9-40da-9fcc-7956c2506a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:09.958728 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53efac8-e7c9-40da-9fcc-7956c2506a99 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:09.968832 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539a749-ab25-4c46-95cf-7177e07d3c87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:09.969658 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539a749-ab25-4c46-95cf-7177e07d3c87 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:09.970081 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539a749-ab25-4c46-95cf-7177e07d3c87 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:09.971356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539a749-ab25-4c46-95cf-7177e07d3c87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:10.158552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539a749-ab25-4c46-95cf-7177e07d3c87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:10.162884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539a749-ab25-4c46-95cf-7177e07d3c87 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:10.179955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1200/2410] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:40:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:40:12.918518 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fa319-0a74-4953-8f4c-f79e90a43105 service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:12.956980 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fa319-0a74-4953-8f4c-f79e90a43105 service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:12.973129 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbce928-ba8f-4297-8f9c-d65053985c00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:12.974655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbce928-ba8f-4297-8f9c-d65053985c00 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:12.975437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbce928-ba8f-4297-8f9c-d65053985c00 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:12.976145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbce928-ba8f-4297-8f9c-d65053985c00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:12.996898 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbce928-ba8f-4297-8f9c-d65053985c00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:13.001239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbce928-ba8f-4297-8f9c-d65053985c00 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:13.031197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1211/2411] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:40:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:40:18.465126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539a749-ab25-4c46-95cf-7177e07d3c87 service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:18.587182 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539a749-ab25-4c46-95cf-7177e07d3c87 service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:18.605126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d382a34-2da8-44a0-9924-27166b98d20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:18.605809 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d382a34-2da8-44a0-9924-27166b98d20e admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:18.606089 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d382a34-2da8-44a0-9924-27166b98d20e admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:18.606365 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d382a34-2da8-44a0-9924-27166b98d20e admin admin] PATH_INFO: `/v3/users/79f64f03029647d0ba88d98500077788` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:18.608738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d382a34-2da8-44a0-9924-27166b98d20e admin admin] RBAC: Authorizing `identity:delete_user(user_id=79f64f03029647d0ba88d98500077788)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:18.610737 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d382a34-2da8-44a0-9924-27166b98d20e admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:18.894126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-0d382a34-2da8-44a0-9924-27166b98d20e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79f64f03029647d0ba88d98500077788'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:18.909275 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-0d382a34-2da8-44a0-9924-27166b98d20e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79f64f03029647d0ba88d98500077788'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:18.910035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-0d382a34-2da8-44a0-9924-27166b98d20e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79f64f03029647d0ba88d98500077788'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:18.912721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:18.919669 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-0d382a34-2da8-44a0-9924-27166b98d20e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79f64f03029647d0ba88d98500077788'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:18.922058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-0d382a34-2da8-44a0-9924-27166b98d20e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79f64f03029647d0ba88d98500077788'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:19.024183 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1201/2412] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:40:18 2022] DELETE /identity/v3/users/79f64f03029647d0ba88d98500077788 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:19.031132 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f8b36-6b38-4967-8fec-bee041f96e15 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:19.031725 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f8b36-6b38-4967-8fec-bee041f96e15 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:19.031725 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f8b36-6b38-4967-8fec-bee041f96e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:19.053576 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-341f8b36-6b38-4967-8fec-bee041f96e15 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:19.057110 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-341f8b36-6b38-4967-8fec-bee041f96e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:19.126624 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1212/2413] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:40:19.137677 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d382a34-2da8-44a0-9924-27166b98d20e admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:19.139869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-0d382a34-2da8-44a0-9924-27166b98d20e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:19.208213 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d382a34-2da8-44a0-9924-27166b98d20e admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:19.212441 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbd551-a18a-4aee-b2f8-b2b925647a7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:19.213083 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbd551-a18a-4aee-b2f8-b2b925647a7f service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:19.213365 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbd551-a18a-4aee-b2f8-b2b925647a7f service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:19.213658 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbd551-a18a-4aee-b2f8-b2b925647a7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:19.229999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbd551-a18a-4aee-b2f8-b2b925647a7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:19.232427 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbd551-a18a-4aee-b2f8-b2b925647a7f service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:19.248344 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1202/2414] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:40:19.802273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f8b36-6b38-4967-8fec-bee041f96e15 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:19.804425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-341f8b36-6b38-4967-8fec-bee041f96e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:19.821383 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f8b36-6b38-4967-8fec-bee041f96e15 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:19.831279 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:19.838558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:19.838558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:19.838558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7 admin admin] PATH_INFO: `/v3/projects/cf4569d8c5784fadb8f0cea716a5d5b3` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:19.838558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf4569d8c5784fadb8f0cea716a5d5b3)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:19.838558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:20.101969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf4569d8c5784fadb8f0cea716a5d5b3', 'request_id': 'req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:20.110447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf4569d8c5784fadb8f0cea716a5d5b3', 'request_id': 'req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:20.340467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf4569d8c5784fadb8f0cea716a5d5b3', 'request_id': 'req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:20.340467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:20.340467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7 admin admin] The token cache is being invalidate because project cf4569d8c5784fadb8f0cea716a5d5b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:40:20.340467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:20.340467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:20.341636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1213/2415] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:40:19 2022] DELETE /identity/v3/projects/cf4569d8c5784fadb8f0cea716a5d5b3 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:20.388608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbd551-a18a-4aee-b2f8-b2b925647a7f service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:20.400049 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbd551-a18a-4aee-b2f8-b2b925647a7f service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:20.480583 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9c33c-0f43-49ec-a1bb-17c9bc14c640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:20.481238 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9c33c-0f43-49ec-a1bb-17c9bc14c640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:20.482364 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9c33c-0f43-49ec-a1bb-17c9bc14c640 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:20.482730 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9c33c-0f43-49ec-a1bb-17c9bc14c640 admin admin] PATH_INFO: `/v3/users/1b9774ebcccb46449b62315cde31c20f` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:20.484162 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e9c33c-0f43-49ec-a1bb-17c9bc14c640 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b9774ebcccb46449b62315cde31c20f)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:20.486152 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e9c33c-0f43-49ec-a1bb-17c9bc14c640 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:20.886360 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a1e9c33c-0f43-49ec-a1bb-17c9bc14c640 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b9774ebcccb46449b62315cde31c20f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:20.903121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a1e9c33c-0f43-49ec-a1bb-17c9bc14c640 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b9774ebcccb46449b62315cde31c20f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:20.904346 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a1e9c33c-0f43-49ec-a1bb-17c9bc14c640 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b9774ebcccb46449b62315cde31c20f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:20.920966 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a1e9c33c-0f43-49ec-a1bb-17c9bc14c640 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b9774ebcccb46449b62315cde31c20f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:20.922284 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a1e9c33c-0f43-49ec-a1bb-17c9bc14c640 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b9774ebcccb46449b62315cde31c20f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:21.054821 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1203/2416] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:40:20 2022] DELETE /identity/v3/users/1b9774ebcccb46449b62315cde31c20f => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:40:21.644324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:21.646717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:21.671636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0f1b3-528d-43ed-9d1d-c0fefeac16a7 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:21.754001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43870e63-16fb-4cda-b765-d1b97d9d1f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:21.754902 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43870e63-16fb-4cda-b765-d1b97d9d1f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:21.755454 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43870e63-16fb-4cda-b765-d1b97d9d1f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:21.755886 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43870e63-16fb-4cda-b765-d1b97d9d1f42 admin admin] PATH_INFO: `/v3/projects/3e0195841981464c9eebdcc54dcb7ecd` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:21.757252 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43870e63-16fb-4cda-b765-d1b97d9d1f42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e0195841981464c9eebdcc54dcb7ecd)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:21.759551 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43870e63-16fb-4cda-b765-d1b97d9d1f42 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:21.978012 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-43870e63-16fb-4cda-b765-d1b97d9d1f42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e0195841981464c9eebdcc54dcb7ecd', 'request_id': 'req-43870e63-16fb-4cda-b765-d1b97d9d1f42'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:21.986249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-43870e63-16fb-4cda-b765-d1b97d9d1f42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e0195841981464c9eebdcc54dcb7ecd', 'request_id': 'req-43870e63-16fb-4cda-b765-d1b97d9d1f42'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:22.067606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696a836-7941-46e8-8299-d89330774e7f None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:22.067606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696a836-7941-46e8-8299-d89330774e7f None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:22.067606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696a836-7941-46e8-8299-d89330774e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:22.080926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:22.088557 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-f696a836-7941-46e8-8299-d89330774e7f None None] MFA Rules not processed for user `8121ba4cf2cf44d9a5a55975ea85928c`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:22.090512 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f696a836-7941-46e8-8299-d89330774e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:22.101046 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-43870e63-16fb-4cda-b765-d1b97d9d1f42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e0195841981464c9eebdcc54dcb7ecd', 'request_id': 'req-43870e63-16fb-4cda-b765-d1b97d9d1f42'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:22.118233 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-43870e63-16fb-4cda-b765-d1b97d9d1f42 admin admin] The token cache is being invalidate because project 3e0195841981464c9eebdcc54dcb7ecd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:40:22.118695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-43870e63-16fb-4cda-b765-d1b97d9d1f42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:22.120100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-43870e63-16fb-4cda-b765-d1b97d9d1f42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:22.123003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1214/2417] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:40:21 2022] DELETE /identity/v3/projects/3e0195841981464c9eebdcc54dcb7ecd => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:22.128666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43870e63-16fb-4cda-b765-d1b97d9d1f42 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:22.130774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-43870e63-16fb-4cda-b765-d1b97d9d1f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:22.199531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1204/2418] 10.222.0.62 () {58 vars in 967 bytes} [Wed Sep 7 13:40:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:22.218446 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696a836-7941-46e8-8299-d89330774e7f None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:22.221015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f696a836-7941-46e8-8299-d89330774e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:22.235841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43870e63-16fb-4cda-b765-d1b97d9d1f42 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:22.243484 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43586ff-85ed-430b-97a0-afa15e0612f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:22.244580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43586ff-85ed-430b-97a0-afa15e0612f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:22.246468 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43586ff-85ed-430b-97a0-afa15e0612f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:22.246468 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43586ff-85ed-430b-97a0-afa15e0612f1 admin admin] PATH_INFO: `/v3/users/03aad8b7cef448a888b48729dbe6cdc9` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:22.248654 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43586ff-85ed-430b-97a0-afa15e0612f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03aad8b7cef448a888b48729dbe6cdc9)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:22.252701 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43586ff-85ed-430b-97a0-afa15e0612f1 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:22.283303 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:22.327264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696a836-7941-46e8-8299-d89330774e7f None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:22.334141 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6b807-8bc4-48ec-8f86-995a1a4476db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cedc7091d2f46b5be68818d26399e9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:22.335070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6b807-8bc4-48ec-8f86-995a1a4476db service swift] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:22.335376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6b807-8bc4-48ec-8f86-995a1a4476db service swift] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:22.335756 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6b807-8bc4-48ec-8f86-995a1a4476db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:22.348341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6b807-8bc4-48ec-8f86-995a1a4476db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:22.350401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6b807-8bc4-48ec-8f86-995a1a4476db service swift] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:22.361790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1205/2419] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 7 13:40:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 13:40:22.559867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e43586ff-85ed-430b-97a0-afa15e0612f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03aad8b7cef448a888b48729dbe6cdc9'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:22.721876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e43586ff-85ed-430b-97a0-afa15e0612f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03aad8b7cef448a888b48729dbe6cdc9'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:22.723369 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e43586ff-85ed-430b-97a0-afa15e0612f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03aad8b7cef448a888b48729dbe6cdc9'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:22.723369 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e43586ff-85ed-430b-97a0-afa15e0612f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03aad8b7cef448a888b48729dbe6cdc9'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:22.734212 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e43586ff-85ed-430b-97a0-afa15e0612f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03aad8b7cef448a888b48729dbe6cdc9'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:22.758490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1215/2420] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:40:22 2022] DELETE /identity/v3/users/03aad8b7cef448a888b48729dbe6cdc9 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:40:23.430224 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6b807-8bc4-48ec-8f86-995a1a4476db service swift] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:23.452613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6b807-8bc4-48ec-8f86-995a1a4476db service swift] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:23.558799 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9e09e-2821-4d18-bd7b-e63c9a65b545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:23.559341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9e09e-2821-4d18-bd7b-e63c9a65b545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:23.560015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9e09e-2821-4d18-bd7b-e63c9a65b545 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:23.560373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9e09e-2821-4d18-bd7b-e63c9a65b545 admin admin] PATH_INFO: `/v3/projects/ad60527e642a41e6a2e6b0b07017b63d` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:23.562058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9e09e-2821-4d18-bd7b-e63c9a65b545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad60527e642a41e6a2e6b0b07017b63d)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:23.564234 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9e09e-2821-4d18-bd7b-e63c9a65b545 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:23.796729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:23.808068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d1a9e09e-2821-4d18-bd7b-e63c9a65b545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad60527e642a41e6a2e6b0b07017b63d', 'request_id': 'req-d1a9e09e-2821-4d18-bd7b-e63c9a65b545'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:23.818902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d1a9e09e-2821-4d18-bd7b-e63c9a65b545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad60527e642a41e6a2e6b0b07017b63d', 'request_id': 'req-d1a9e09e-2821-4d18-bd7b-e63c9a65b545'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:24.001156 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d1a9e09e-2821-4d18-bd7b-e63c9a65b545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad60527e642a41e6a2e6b0b07017b63d', 'request_id': 'req-d1a9e09e-2821-4d18-bd7b-e63c9a65b545'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:24.015996 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d1a9e09e-2821-4d18-bd7b-e63c9a65b545 admin admin] The token cache is being invalidate because project ad60527e642a41e6a2e6b0b07017b63d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:40:24.016501 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d1a9e09e-2821-4d18-bd7b-e63c9a65b545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:24.017627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d1a9e09e-2821-4d18-bd7b-e63c9a65b545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:24.020369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1206/2421] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:40:23 2022] DELETE /identity/v3/projects/ad60527e642a41e6a2e6b0b07017b63d => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:24.028570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2edad5-c08e-4252-aff9-67f6a53882ab None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:24.029306 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2edad5-c08e-4252-aff9-67f6a53882ab None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:24.029623 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2edad5-c08e-4252-aff9-67f6a53882ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:24.066861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-1d2edad5-c08e-4252-aff9-67f6a53882ab None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:24.068983 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1d2edad5-c08e-4252-aff9-67f6a53882ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:24.164905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1216/2422] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:24.174121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c03ded-7119-4f80-8f23-e447686640ef None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:24.174456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c03ded-7119-4f80-8f23-e447686640ef None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:24.174768 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c03ded-7119-4f80-8f23-e447686640ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:24.207664 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-75c03ded-7119-4f80-8f23-e447686640ef None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:24.209004 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-75c03ded-7119-4f80-8f23-e447686640ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:24.220472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1207/2423] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:24.225243 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2edad5-c08e-4252-aff9-67f6a53882ab None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:24.237289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2edad5-c08e-4252-aff9-67f6a53882ab None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:24.242751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0e9ef-c640-47d5-bfc1-b6a049c68dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:24.243432 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0e9ef-c640-47d5-bfc1-b6a049c68dcf admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:24.243714 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0e9ef-c640-47d5-bfc1-b6a049c68dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:24.243988 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0e9ef-c640-47d5-bfc1-b6a049c68dcf admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:24.244726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf0e9ef-c640-47d5-bfc1-b6a049c68dcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:40:24.245033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf0e9ef-c640-47d5-bfc1-b6a049c68dcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:24.246761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf0e9ef-c640-47d5-bfc1-b6a049c68dcf admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:24.262688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1217/2424] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:40:24.269010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c03ded-7119-4f80-8f23-e447686640ef None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:24.287289 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c03ded-7119-4f80-8f23-e447686640ef None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:24.295628 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2215bd3-ee45-4809-99b9-545e03e23e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:24.296702 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2215bd3-ee45-4809-99b9-545e03e23e81 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:24.297188 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2215bd3-ee45-4809-99b9-545e03e23e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:24.297720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2215bd3-ee45-4809-99b9-545e03e23e81 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:24.298932 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2215bd3-ee45-4809-99b9-545e03e23e81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:24.302587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2215bd3-ee45-4809-99b9-545e03e23e81 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:24.413414 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1208/2425] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:40:24 2022] POST /identity/v3/projects => generated 367 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:40:24.425923 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0e9ef-c640-47d5-bfc1-b6a049c68dcf admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:24.442860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0e9ef-c640-47d5-bfc1-b6a049c68dcf admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:24.532220 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad5a43-202f-4f48-afc0-23c850a71b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:24.533251 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad5a43-202f-4f48-afc0-23c850a71b1e admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:24.533727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad5a43-202f-4f48-afc0-23c850a71b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:24.534497 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad5a43-202f-4f48-afc0-23c850a71b1e admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:24.535935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad5a43-202f-4f48-afc0-23c850a71b1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:24.538419 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad5a43-202f-4f48-afc0-23c850a71b1e admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:24.644424 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1218/2426] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:40:24 2022] POST /identity/v3/users => generated 284 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:40:24.649355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2215bd3-ee45-4809-99b9-545e03e23e81 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:24.661839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2215bd3-ee45-4809-99b9-545e03e23e81 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:24.667188 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b609e-44cf-4188-ae9e-633f8d618f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:24.667816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b609e-44cf-4188-ae9e-633f8d618f08 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:24.668101 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b609e-44cf-4188-ae9e-633f8d618f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:24.668373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b609e-44cf-4188-ae9e-633f8d618f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:24.669747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b609e-44cf-4188-ae9e-633f8d618f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:24.671480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b609e-44cf-4188-ae9e-633f8d618f08 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:24.683488 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1209/2427] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:40:24.688427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad5a43-202f-4f48-afc0-23c850a71b1e admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:24.700095 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad5a43-202f-4f48-afc0-23c850a71b1e admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:24.705505 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188da50-9744-40cb-a39c-8dbe02ad2cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:24.706255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188da50-9744-40cb-a39c-8dbe02ad2cac admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:24.706621 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188da50-9744-40cb-a39c-8dbe02ad2cac admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:24.706901 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188da50-9744-40cb-a39c-8dbe02ad2cac admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:24.707603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188da50-9744-40cb-a39c-8dbe02ad2cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:24.709249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188da50-9744-40cb-a39c-8dbe02ad2cac admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:24.724920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1219/2428] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:40:24.729526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b609e-44cf-4188-ae9e-633f8d618f08 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:24.742067 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b609e-44cf-4188-ae9e-633f8d618f08 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:24.752834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdebb9-f90e-4044-b767-8b758d797a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:24.753609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdebb9-f90e-4044-b767-8b758d797a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:24.753889 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdebb9-f90e-4044-b767-8b758d797a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:24.754217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdebb9-f90e-4044-b767-8b758d797a70 admin admin] PATH_INFO: `/v3/projects/0af1f942d50647648e712882c3bb7909/users/4c51481b58254cf49831a865f3d46579/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:24.775140 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fdebb9-f90e-4044-b767-8b758d797a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af1f942d50647648e712882c3bb7909, user_id=4c51481b58254cf49831a865f3d46579, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:24.782114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fdebb9-f90e-4044-b767-8b758d797a70 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:24.924764 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1210/2429] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:40:24 2022] PUT /identity/v3/projects/0af1f942d50647648e712882c3bb7909/users/4c51481b58254cf49831a865f3d46579/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:25.002527 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914d6ff-47d6-4983-91f1-bd010508ae10 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:25.003119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914d6ff-47d6-4983-91f1-bd010508ae10 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:25.003615 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914d6ff-47d6-4983-91f1-bd010508ae10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:25.019080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:25.058263 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-c914d6ff-47d6-4983-91f1-bd010508ae10 None None] MFA Rules not processed for user `4c51481b58254cf49831a865f3d46579`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:25.061283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c914d6ff-47d6-4983-91f1-bd010508ae10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:25.187174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1220/2430] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:25.199864 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdebb9-f90e-4044-b767-8b758d797a70 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:25.209339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-57fdebb9-f90e-4044-b767-8b758d797a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:25.296658 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdebb9-f90e-4044-b767-8b758d797a70 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:25.303806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e71f07-217b-4acd-ae18-b7287ed9d320 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1a48805d95640489d7ce3333dd687b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:25.305152 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e71f07-217b-4acd-ae18-b7287ed9d320 service glance] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:25.305599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e71f07-217b-4acd-ae18-b7287ed9d320 service glance] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:25.306178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e71f07-217b-4acd-ae18-b7287ed9d320 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:25.323921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e71f07-217b-4acd-ae18-b7287ed9d320 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:25.326803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e71f07-217b-4acd-ae18-b7287ed9d320 service glance] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:25.345394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1211/2431] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:40:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:40:32.382069 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914d6ff-47d6-4983-91f1-bd010508ae10 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:32.384385 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c914d6ff-47d6-4983-91f1-bd010508ae10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:32.564327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914d6ff-47d6-4983-91f1-bd010508ae10 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:32.574008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fca282-9b4a-40da-9c17-cc99c1ed84d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bee90bf951f04c2e8e19ceeacb8cfcb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:32.576051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fca282-9b4a-40da-9c17-cc99c1ed84d2 service placement] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:32.576051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fca282-9b4a-40da-9c17-cc99c1ed84d2 service placement] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:32.576929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fca282-9b4a-40da-9c17-cc99c1ed84d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:32.580465 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-28fca282-9b4a-40da-9c17-cc99c1ed84d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:32.603425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fca282-9b4a-40da-9c17-cc99c1ed84d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:32.607118 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fca282-9b4a-40da-9c17-cc99c1ed84d2 service placement] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:32.628662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1221/2432] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:40:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:40:35.336477 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=64,keystone:DELETE=4,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:35.493167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e71f07-217b-4acd-ae18-b7287ed9d320 service glance] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:35.495645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c0e71f07-217b-4acd-ae18-b7287ed9d320 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:35.619422 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e71f07-217b-4acd-ae18-b7287ed9d320 service glance] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:35.626750 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fadd0-d1d7-42c8-aa67-9e9336cabf9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:35.627997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fadd0-d1d7-42c8-aa67-9e9336cabf9c service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:35.628492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fadd0-d1d7-42c8-aa67-9e9336cabf9c service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:35.628981 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fadd0-d1d7-42c8-aa67-9e9336cabf9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:35.632470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a03fadd0-d1d7-42c8-aa67-9e9336cabf9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:35.735166 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fadd0-d1d7-42c8-aa67-9e9336cabf9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:35.736919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fadd0-d1d7-42c8-aa67-9e9336cabf9c service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:35.750100 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1212/2433] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:40:40.938320 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fca282-9b4a-40da-9c17-cc99c1ed84d2 service placement] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:40.942807 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-28fca282-9b4a-40da-9c17-cc99c1ed84d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:40.955298 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fca282-9b4a-40da-9c17-cc99c1ed84d2 service placement] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:40.961172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b5fd3-30bc-4b7c-9e8c-54359ada6cd2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bee90bf951f04c2e8e19ceeacb8cfcb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:40.961824 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b5fd3-30bc-4b7c-9e8c-54359ada6cd2 service placement] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:40.962090 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b5fd3-30bc-4b7c-9e8c-54359ada6cd2 service placement] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:40.963117 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b5fd3-30bc-4b7c-9e8c-54359ada6cd2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:40.966069 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-590b5fd3-30bc-4b7c-9e8c-54359ada6cd2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:40.982135 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b5fd3-30bc-4b7c-9e8c-54359ada6cd2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:40.984762 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b5fd3-30bc-4b7c-9e8c-54359ada6cd2 service placement] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:41.031138 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1222/2434] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:40:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:40:41.144983 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fadd0-d1d7-42c8-aa67-9e9336cabf9c service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:41.148609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a03fadd0-d1d7-42c8-aa67-9e9336cabf9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:41.173177 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fadd0-d1d7-42c8-aa67-9e9336cabf9c service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:41.181299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e040b-2106-46be-9530-1e36420d7fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:41.182346 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e040b-2106-46be-9530-1e36420d7fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:41.182826 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e040b-2106-46be-9530-1e36420d7fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:41.183370 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e040b-2106-46be-9530-1e36420d7fbd admin admin] PATH_INFO: `/v3/users/7e7db29f64934026abbe96d5cb5baf25` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:41.185579 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e040b-2106-46be-9530-1e36420d7fbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e7db29f64934026abbe96d5cb5baf25)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:41.189385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e040b-2106-46be-9530-1e36420d7fbd admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:41.454524 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-029e040b-2106-46be-9530-1e36420d7fbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e7db29f64934026abbe96d5cb5baf25'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:41.472522 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-029e040b-2106-46be-9530-1e36420d7fbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e7db29f64934026abbe96d5cb5baf25'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:41.473519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-029e040b-2106-46be-9530-1e36420d7fbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e7db29f64934026abbe96d5cb5baf25'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:41.489121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-029e040b-2106-46be-9530-1e36420d7fbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e7db29f64934026abbe96d5cb5baf25'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:41.490230 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-029e040b-2106-46be-9530-1e36420d7fbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e7db29f64934026abbe96d5cb5baf25'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:41.594165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1213/2435] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:40:41 2022] DELETE /identity/v3/users/7e7db29f64934026abbe96d5cb5baf25 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:41.969028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b5fd3-30bc-4b7c-9e8c-54359ada6cd2 service placement] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:41.982445 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b5fd3-30bc-4b7c-9e8c-54359ada6cd2 service placement] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:42.058872 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:42.076607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36a9d9-0e09-4bc1-9323-629c053fdf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:42.077819 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36a9d9-0e09-4bc1-9323-629c053fdf51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:42.078304 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36a9d9-0e09-4bc1-9323-629c053fdf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:42.078760 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36a9d9-0e09-4bc1-9323-629c053fdf51 admin admin] PATH_INFO: `/v3/projects/e0e892aa9df049c387360b67c5e2e065` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:42.081090 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36a9d9-0e09-4bc1-9323-629c053fdf51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0e892aa9df049c387360b67c5e2e065)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:42.084398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b36a9d9-0e09-4bc1-9323-629c053fdf51 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:42.217208 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9b36a9d9-0e09-4bc1-9323-629c053fdf51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0e892aa9df049c387360b67c5e2e065', 'request_id': 'req-9b36a9d9-0e09-4bc1-9323-629c053fdf51'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:42.228188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9b36a9d9-0e09-4bc1-9323-629c053fdf51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0e892aa9df049c387360b67c5e2e065', 'request_id': 'req-9b36a9d9-0e09-4bc1-9323-629c053fdf51'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:42.311179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9b36a9d9-0e09-4bc1-9323-629c053fdf51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0e892aa9df049c387360b67c5e2e065', 'request_id': 'req-9b36a9d9-0e09-4bc1-9323-629c053fdf51'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:42.326425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9b36a9d9-0e09-4bc1-9323-629c053fdf51 admin admin] The token cache is being invalidate because project e0e892aa9df049c387360b67c5e2e065 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:40:42.326891 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9b36a9d9-0e09-4bc1-9323-629c053fdf51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:42.327748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-9b36a9d9-0e09-4bc1-9323-629c053fdf51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:42.335095 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1223/2436] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:40:41 2022] DELETE /identity/v3/projects/e0e892aa9df049c387360b67c5e2e065 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:42.340337 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f5283-7960-492c-aec3-844be1660706 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:42.341220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f5283-7960-492c-aec3-844be1660706 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:42.341220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f5283-7960-492c-aec3-844be1660706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:42.376645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c25f5283-7960-492c-aec3-844be1660706 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:42.376645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c25f5283-7960-492c-aec3-844be1660706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:42.398578 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:42.461565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1214/2437] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:42.475215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d67e9-3aff-40e3-8ea6-323e9fe854a4 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:42.475898 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d67e9-3aff-40e3-8ea6-323e9fe854a4 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:42.476488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d67e9-3aff-40e3-8ea6-323e9fe854a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:42.503907 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-f79d67e9-3aff-40e3-8ea6-323e9fe854a4 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:42.505367 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f79d67e9-3aff-40e3-8ea6-323e9fe854a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:42.516358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1224/2438] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:42.522015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f5283-7960-492c-aec3-844be1660706 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:42.539726 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f5283-7960-492c-aec3-844be1660706 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:42.548901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6619ea8-b896-4e43-968b-c07bf1a8067c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:42.549894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6619ea8-b896-4e43-968b-c07bf1a8067c admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:42.550408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6619ea8-b896-4e43-968b-c07bf1a8067c admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:42.550779 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6619ea8-b896-4e43-968b-c07bf1a8067c admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:42.551919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6619ea8-b896-4e43-968b-c07bf1a8067c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:40:42.552420 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6619ea8-b896-4e43-968b-c07bf1a8067c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:42.555301 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6619ea8-b896-4e43-968b-c07bf1a8067c admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:42.572262 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1215/2439] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:40:42.590053 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d67e9-3aff-40e3-8ea6-323e9fe854a4 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:42.609238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d67e9-3aff-40e3-8ea6-323e9fe854a4 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:42.617753 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba7bb2-04b3-42a9-8ded-72d3149f7468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:42.618479 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba7bb2-04b3-42a9-8ded-72d3149f7468 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:42.618630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba7bb2-04b3-42a9-8ded-72d3149f7468 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:42.618900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba7bb2-04b3-42a9-8ded-72d3149f7468 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:42.619591 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba7bb2-04b3-42a9-8ded-72d3149f7468 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:42.622051 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba7bb2-04b3-42a9-8ded-72d3149f7468 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:42.733249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1225/2440] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:40:42 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:40:42.735277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6619ea8-b896-4e43-968b-c07bf1a8067c admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:42.754595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6619ea8-b896-4e43-968b-c07bf1a8067c admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:42.842438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0cd984-441f-4f77-9873-a4ca1751e60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:42.843109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cd984-441f-4f77-9873-a4ca1751e60b admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:42.843398 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cd984-441f-4f77-9873-a4ca1751e60b admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:42.844028 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cd984-441f-4f77-9873-a4ca1751e60b admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:42.844695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0cd984-441f-4f77-9873-a4ca1751e60b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:42.846775 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0cd984-441f-4f77-9873-a4ca1751e60b admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:42.987901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1216/2441] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:40:42 2022] POST /identity/v3/users => generated 287 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:40:42.993876 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba7bb2-04b3-42a9-8ded-72d3149f7468 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:43.007622 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba7bb2-04b3-42a9-8ded-72d3149f7468 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:43.013831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9cb25-827e-40b6-b2b2-05b949e6fe24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:43.014584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9cb25-827e-40b6-b2b2-05b949e6fe24 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:43.014992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9cb25-827e-40b6-b2b2-05b949e6fe24 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:43.015325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9cb25-827e-40b6-b2b2-05b949e6fe24 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:43.016125 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9cb25-827e-40b6-b2b2-05b949e6fe24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:43.018230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9cb25-827e-40b6-b2b2-05b949e6fe24 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:43.032996 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1226/2442] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:40:43.039559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0cd984-441f-4f77-9873-a4ca1751e60b admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:43.055010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0cd984-441f-4f77-9873-a4ca1751e60b admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:43.061350 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d39e43-91f5-4f04-976a-52211d7bf150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:43.065132 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d39e43-91f5-4f04-976a-52211d7bf150 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:43.065132 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d39e43-91f5-4f04-976a-52211d7bf150 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:43.065132 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d39e43-91f5-4f04-976a-52211d7bf150 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:43.065132 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d39e43-91f5-4f04-976a-52211d7bf150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:43.065738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d39e43-91f5-4f04-976a-52211d7bf150 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:43.081193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1217/2443] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:40:43.086690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9cb25-827e-40b6-b2b2-05b949e6fe24 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:43.102883 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9cb25-827e-40b6-b2b2-05b949e6fe24 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:43.112196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a11b3-3b97-4621-8d6f-7562bd963749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:43.113983 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a11b3-3b97-4621-8d6f-7562bd963749 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:43.114425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a11b3-3b97-4621-8d6f-7562bd963749 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:43.114792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a11b3-3b97-4621-8d6f-7562bd963749 admin admin] PATH_INFO: `/v3/projects/272e8d8d92e54e14854aac920bb29ebf/users/ea8d8f20806d42d9952906ea73d50642/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:43.144529 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a11b3-3b97-4621-8d6f-7562bd963749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=272e8d8d92e54e14854aac920bb29ebf, user_id=ea8d8f20806d42d9952906ea73d50642, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:43.150930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a11b3-3b97-4621-8d6f-7562bd963749 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:43.295652 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1227/2444] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:40:43 2022] PUT /identity/v3/projects/272e8d8d92e54e14854aac920bb29ebf/users/ea8d8f20806d42d9952906ea73d50642/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:43.308005 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f4a73-2611-44a8-9b47-f6d50a89ea6f None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:43.309547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f4a73-2611-44a8-9b47-f6d50a89ea6f None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:43.310313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f4a73-2611-44a8-9b47-f6d50a89ea6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:43.346201 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-0c8f4a73-2611-44a8-9b47-f6d50a89ea6f None None] MFA Rules not processed for user `ea8d8f20806d42d9952906ea73d50642`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:43.347502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-0c8f4a73-2611-44a8-9b47-f6d50a89ea6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:43.423881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1218/2445] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:43.436306 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a11b3-3b97-4621-8d6f-7562bd963749 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:43.438151 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a15a11b3-3b97-4621-8d6f-7562bd963749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:43.533652 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:43.553461 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a11b3-3b97-4621-8d6f-7562bd963749 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:43.568562 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777285e5-4925-414a-abd6-dba69492706a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:43.569311 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777285e5-4925-414a-abd6-dba69492706a service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:43.569644 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777285e5-4925-414a-abd6-dba69492706a service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:43.570092 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777285e5-4925-414a-abd6-dba69492706a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:43.580065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777285e5-4925-414a-abd6-dba69492706a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:43.581724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777285e5-4925-414a-abd6-dba69492706a service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:43.592305 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1228/2446] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:40:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:40:50.548751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f4a73-2611-44a8-9b47-f6d50a89ea6f None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:50.578106 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:50.637957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f4a73-2611-44a8-9b47-f6d50a89ea6f None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:50.643967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783456b4-f798-4c83-b590-a8a34a4d87bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:50.644627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783456b4-f798-4c83-b590-a8a34a4d87bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:50.645046 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783456b4-f798-4c83-b590-a8a34a4d87bc admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:50.645228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783456b4-f798-4c83-b590-a8a34a4d87bc admin admin] PATH_INFO: `/v3/users/ea8d8f20806d42d9952906ea73d50642` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:50.646633 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783456b4-f798-4c83-b590-a8a34a4d87bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea8d8f20806d42d9952906ea73d50642)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:50.649135 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783456b4-f798-4c83-b590-a8a34a4d87bc admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:50.918354 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-783456b4-f798-4c83-b590-a8a34a4d87bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea8d8f20806d42d9952906ea73d50642'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:50.946419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-783456b4-f798-4c83-b590-a8a34a4d87bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea8d8f20806d42d9952906ea73d50642'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:50.947869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-783456b4-f798-4c83-b590-a8a34a4d87bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea8d8f20806d42d9952906ea73d50642'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:50.966132 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-783456b4-f798-4c83-b590-a8a34a4d87bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea8d8f20806d42d9952906ea73d50642'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:50.967456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-783456b4-f798-4c83-b590-a8a34a4d87bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea8d8f20806d42d9952906ea73d50642'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:51.069955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1219/2447] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:40:50 2022] DELETE /identity/v3/users/ea8d8f20806d42d9952906ea73d50642 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:40:51.076705 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a0218-38f7-402f-ba64-f66d31d0af36 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:51.077580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a0218-38f7-402f-ba64-f66d31d0af36 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:51.077580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a0218-38f7-402f-ba64-f66d31d0af36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:51.103141 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-583a0218-38f7-402f-ba64-f66d31d0af36 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:51.104805 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-583a0218-38f7-402f-ba64-f66d31d0af36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:51.193345 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1229/2448] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:51.205582 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783456b4-f798-4c83-b590-a8a34a4d87bc admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:51.207371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-783456b4-f798-4c83-b590-a8a34a4d87bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:51.313921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783456b4-f798-4c83-b590-a8a34a4d87bc admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:51.320026 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881efe1a-d463-4b61-81a4-a54052895824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:51.321047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881efe1a-d463-4b61-81a4-a54052895824 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:51.321332 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881efe1a-d463-4b61-81a4-a54052895824 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:51.321605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881efe1a-d463-4b61-81a4-a54052895824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:51.337171 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881efe1a-d463-4b61-81a4-a54052895824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:51.340615 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881efe1a-d463-4b61-81a4-a54052895824 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:51.356759 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1220/2449] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:40:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:40:52.020045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a0218-38f7-402f-ba64-f66d31d0af36 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:52.022644 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-583a0218-38f7-402f-ba64-f66d31d0af36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:52.068942 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a0218-38f7-402f-ba64-f66d31d0af36 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:52.077566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c19fc-4cde-467d-aef4-687c05bb5103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:52.079215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c19fc-4cde-467d-aef4-687c05bb5103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:52.079610 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c19fc-4cde-467d-aef4-687c05bb5103 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:52.080501 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c19fc-4cde-467d-aef4-687c05bb5103 admin admin] PATH_INFO: `/v3/projects/272e8d8d92e54e14854aac920bb29ebf` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:52.082664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c19fc-4cde-467d-aef4-687c05bb5103 admin admin] RBAC: Authorizing `identity:delete_project(project_id=272e8d8d92e54e14854aac920bb29ebf)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:52.085115 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c19fc-4cde-467d-aef4-687c05bb5103 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:52.324635 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-706c19fc-4cde-467d-aef4-687c05bb5103 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '272e8d8d92e54e14854aac920bb29ebf', 'request_id': 'req-706c19fc-4cde-467d-aef4-687c05bb5103'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:52.345040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-706c19fc-4cde-467d-aef4-687c05bb5103 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '272e8d8d92e54e14854aac920bb29ebf', 'request_id': 'req-706c19fc-4cde-467d-aef4-687c05bb5103'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:52.529270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-706c19fc-4cde-467d-aef4-687c05bb5103 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '272e8d8d92e54e14854aac920bb29ebf', 'request_id': 'req-706c19fc-4cde-467d-aef4-687c05bb5103'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:52.550255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-706c19fc-4cde-467d-aef4-687c05bb5103 admin admin] The token cache is being invalidate because project 272e8d8d92e54e14854aac920bb29ebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:40:52.550605 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-706c19fc-4cde-467d-aef4-687c05bb5103 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:52.551424 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-706c19fc-4cde-467d-aef4-687c05bb5103 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:52.553960 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1230/2450] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:40:52 2022] DELETE /identity/v3/projects/272e8d8d92e54e14854aac920bb29ebf => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:52.560597 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47cdc53-deaa-4d21-98ba-e22d080bdbdc None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:52.562171 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47cdc53-deaa-4d21-98ba-e22d080bdbdc None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:52.562947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47cdc53-deaa-4d21-98ba-e22d080bdbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:52.633412 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-a47cdc53-deaa-4d21-98ba-e22d080bdbdc None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:52.636024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a47cdc53-deaa-4d21-98ba-e22d080bdbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:52.726827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:52.788299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1221/2451] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:52.803784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb8064-131d-4f31-b7f3-3d756a6923e7 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:52.804103 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb8064-131d-4f31-b7f3-3d756a6923e7 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:52.805765 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb8064-131d-4f31-b7f3-3d756a6923e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:52.854332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-c3fb8064-131d-4f31-b7f3-3d756a6923e7 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:52.856964 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c3fb8064-131d-4f31-b7f3-3d756a6923e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:52.867260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1231/2452] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:52.874792 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47cdc53-deaa-4d21-98ba-e22d080bdbdc None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:52.887396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47cdc53-deaa-4d21-98ba-e22d080bdbdc None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:52.896590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c2e61-63a4-4acd-9762-6a5f148c7126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:52.898816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c2e61-63a4-4acd-9762-6a5f148c7126 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:52.899382 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c2e61-63a4-4acd-9762-6a5f148c7126 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:52.899870 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c2e61-63a4-4acd-9762-6a5f148c7126 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:52.901279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c2e61-63a4-4acd-9762-6a5f148c7126 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:40:52.901807 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c2e61-63a4-4acd-9762-6a5f148c7126 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:52.905357 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c2e61-63a4-4acd-9762-6a5f148c7126 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:52.925609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1222/2453] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:40:52.930787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb8064-131d-4f31-b7f3-3d756a6923e7 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:52.952160 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb8064-131d-4f31-b7f3-3d756a6923e7 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:52.962820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387d9a1-d79b-4066-8485-18907c62280d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:52.964382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387d9a1-d79b-4066-8485-18907c62280d admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:52.965235 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387d9a1-d79b-4066-8485-18907c62280d admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:52.966866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387d9a1-d79b-4066-8485-18907c62280d admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:52.969998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387d9a1-d79b-4066-8485-18907c62280d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:52.975284 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387d9a1-d79b-4066-8485-18907c62280d admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:53.146938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1232/2454] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:40:52 2022] POST /identity/v3/projects => generated 403 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:40:53.150935 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c2e61-63a4-4acd-9762-6a5f148c7126 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:53.174461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c2e61-63a4-4acd-9762-6a5f148c7126 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:53.283710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166a496-c828-4e51-812e-5835e03a03bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:53.285723 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166a496-c828-4e51-812e-5835e03a03bd admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:53.285723 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166a496-c828-4e51-812e-5835e03a03bd admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:53.285723 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166a496-c828-4e51-812e-5835e03a03bd admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:53.286453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166a496-c828-4e51-812e-5835e03a03bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:53.289622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166a496-c828-4e51-812e-5835e03a03bd admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:53.438155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1223/2455] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:40:53 2022] POST /identity/v3/users => generated 302 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:40:53.443333 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387d9a1-d79b-4066-8485-18907c62280d admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:53.462565 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387d9a1-d79b-4066-8485-18907c62280d admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:53.471892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e873cc9-b4c6-4b8c-bb0c-9126959a31e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:53.472790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e873cc9-b4c6-4b8c-bb0c-9126959a31e6 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:53.473347 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e873cc9-b4c6-4b8c-bb0c-9126959a31e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:53.473730 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e873cc9-b4c6-4b8c-bb0c-9126959a31e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:53.474777 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e873cc9-b4c6-4b8c-bb0c-9126959a31e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:53.477990 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e873cc9-b4c6-4b8c-bb0c-9126959a31e6 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:53.499131 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1233/2456] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:40:53.504915 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166a496-c828-4e51-812e-5835e03a03bd admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:53.518372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166a496-c828-4e51-812e-5835e03a03bd admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:53.530797 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d535467-05f0-4a62-950e-5cd2ad451f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:53.531082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d535467-05f0-4a62-950e-5cd2ad451f86 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:53.531339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d535467-05f0-4a62-950e-5cd2ad451f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:53.534601 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d535467-05f0-4a62-950e-5cd2ad451f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:53.535560 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d535467-05f0-4a62-950e-5cd2ad451f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:53.537388 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d535467-05f0-4a62-950e-5cd2ad451f86 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:53.562325 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1224/2457] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:40:53.563362 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e873cc9-b4c6-4b8c-bb0c-9126959a31e6 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:53.578857 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e873cc9-b4c6-4b8c-bb0c-9126959a31e6 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:53.585149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a2e9a-3bc4-4bde-aeb4-d2214310d737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:53.585935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a2e9a-3bc4-4bde-aeb4-d2214310d737 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:53.586211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a2e9a-3bc4-4bde-aeb4-d2214310d737 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:53.586493 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a2e9a-3bc4-4bde-aeb4-d2214310d737 admin admin] PATH_INFO: `/v3/projects/e019ccc8d69c4238940e7d3a7c3ce5d3/users/4a2044bcebbf47e791dc004f2ed2800d/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:53.607952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a2e9a-3bc4-4bde-aeb4-d2214310d737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e019ccc8d69c4238940e7d3a7c3ce5d3, user_id=4a2044bcebbf47e791dc004f2ed2800d, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:53.610701 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a2e9a-3bc4-4bde-aeb4-d2214310d737 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:53.732600 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1234/2458] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:40:53 2022] PUT /identity/v3/projects/e019ccc8d69c4238940e7d3a7c3ce5d3/users/4a2044bcebbf47e791dc004f2ed2800d/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:53.748005 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1101162-4b34-4dd8-b907-e300cb5d867b None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:53.748292 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1101162-4b34-4dd8-b907-e300cb5d867b None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:53.748574 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1101162-4b34-4dd8-b907-e300cb5d867b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:53.790192 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-e1101162-4b34-4dd8-b907-e300cb5d867b None None] MFA Rules not processed for user `4a2044bcebbf47e791dc004f2ed2800d`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:53.791986 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e1101162-4b34-4dd8-b907-e300cb5d867b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:53.947313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1225/2459] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:53 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:53.949889 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a2e9a-3bc4-4bde-aeb4-d2214310d737 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:53.955349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:54.037192 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a2e9a-3bc4-4bde-aeb4-d2214310d737 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:54.042816 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c745a-e93f-41fd-8f3e-fdf79bcf2f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:54.043538 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c745a-e93f-41fd-8f3e-fdf79bcf2f33 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:54.043846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c745a-e93f-41fd-8f3e-fdf79bcf2f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:54.044390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c745a-e93f-41fd-8f3e-fdf79bcf2f33 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:54.045395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c745a-e93f-41fd-8f3e-fdf79bcf2f33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:54.048461 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c745a-e93f-41fd-8f3e-fdf79bcf2f33 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:54.174300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1235/2460] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:40:53 2022] POST /identity/v3/projects => generated 403 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:40:54.178079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1101162-4b34-4dd8-b907-e300cb5d867b None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:54.191871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1101162-4b34-4dd8-b907-e300cb5d867b None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:54.265379 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd382abf-ded9-4fc6-bc7e-50f3050583e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:54.266142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd382abf-ded9-4fc6-bc7e-50f3050583e9 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:54.266465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd382abf-ded9-4fc6-bc7e-50f3050583e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:54.266773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd382abf-ded9-4fc6-bc7e-50f3050583e9 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:54.267574 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd382abf-ded9-4fc6-bc7e-50f3050583e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:54.269896 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd382abf-ded9-4fc6-bc7e-50f3050583e9 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:54.376002 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1226/2461] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:40:54 2022] POST /identity/v3/users => generated 308 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:40:54.380955 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c745a-e93f-41fd-8f3e-fdf79bcf2f33 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:54.393610 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c745a-e93f-41fd-8f3e-fdf79bcf2f33 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:54.399823 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb9cd6-68bd-4397-b3c4-e6e7a589c0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:54.403867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb9cd6-68bd-4397-b3c4-e6e7a589c0a2 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:54.403867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb9cd6-68bd-4397-b3c4-e6e7a589c0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:54.403867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb9cd6-68bd-4397-b3c4-e6e7a589c0a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:54.403867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb9cd6-68bd-4397-b3c4-e6e7a589c0a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:54.404586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb9cd6-68bd-4397-b3c4-e6e7a589c0a2 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:54.421082 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1236/2462] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:40:54.426031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd382abf-ded9-4fc6-bc7e-50f3050583e9 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:54.439147 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd382abf-ded9-4fc6-bc7e-50f3050583e9 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:54.444729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb25955-d143-45be-96f2-03ab609cef98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:54.445527 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb25955-d143-45be-96f2-03ab609cef98 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:54.445912 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb25955-d143-45be-96f2-03ab609cef98 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:54.446233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb25955-d143-45be-96f2-03ab609cef98 admin admin] PATH_INFO: `/v3/projects/40cf13ab45994aba996fae12564c3c63/users/5962c6330fe84206b0408f1414124c2a/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:54.466449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb25955-d143-45be-96f2-03ab609cef98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40cf13ab45994aba996fae12564c3c63, user_id=5962c6330fe84206b0408f1414124c2a, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:54.470709 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb25955-d143-45be-96f2-03ab609cef98 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:54.562932 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:54.603009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1227/2463] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:40:54 2022] PUT /identity/v3/projects/40cf13ab45994aba996fae12564c3c63/users/5962c6330fe84206b0408f1414124c2a/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:54.607417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb9cd6-68bd-4397-b3c4-e6e7a589c0a2 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:54.618613 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb9cd6-68bd-4397-b3c4-e6e7a589c0a2 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:54.713718 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d0b66-fd69-48cf-8ad8-55c590d810cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:54.714216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d0b66-fd69-48cf-8ad8-55c590d810cd admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:54.715119 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d0b66-fd69-48cf-8ad8-55c590d810cd admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:54.715532 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d0b66-fd69-48cf-8ad8-55c590d810cd admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:54.716000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d0b66-fd69-48cf-8ad8-55c590d810cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:54.717750 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d0b66-fd69-48cf-8ad8-55c590d810cd admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:54.730223 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1237/2464] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:40:54.737256 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb25955-d143-45be-96f2-03ab609cef98 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:54.750607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb25955-d143-45be-96f2-03ab609cef98 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:54.756080 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809eff1e-a243-44d8-a640-def85ff5750f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:54.756797 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809eff1e-a243-44d8-a640-def85ff5750f admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:54.757076 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809eff1e-a243-44d8-a640-def85ff5750f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:54.757353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809eff1e-a243-44d8-a640-def85ff5750f admin admin] PATH_INFO: `/v3/projects/40cf13ab45994aba996fae12564c3c63/users/5962c6330fe84206b0408f1414124c2a/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:54.759425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809eff1e-a243-44d8-a640-def85ff5750f admin admin] RBAC: Authorizing `identity:create_grant(project_id=40cf13ab45994aba996fae12564c3c63, user_id=5962c6330fe84206b0408f1414124c2a, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:54.763109 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809eff1e-a243-44d8-a640-def85ff5750f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:54.775746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1228/2465] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:40:54 2022] PUT /identity/v3/projects/40cf13ab45994aba996fae12564c3c63/users/5962c6330fe84206b0408f1414124c2a/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:54.797712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008d54e-661b-4e9f-a6ad-d5374d7ec7b5 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:54.806385 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008d54e-661b-4e9f-a6ad-d5374d7ec7b5 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:54.806385 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008d54e-661b-4e9f-a6ad-d5374d7ec7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:54.850654 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-7008d54e-661b-4e9f-a6ad-d5374d7ec7b5 None None] MFA Rules not processed for user `5962c6330fe84206b0408f1414124c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:54.853046 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7008d54e-661b-4e9f-a6ad-d5374d7ec7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:54.993737 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1238/2466] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:54 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:54.996506 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:55.010347 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809eff1e-a243-44d8-a640-def85ff5750f admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:55.014341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-809eff1e-a243-44d8-a640-def85ff5750f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:55.120419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809eff1e-a243-44d8-a640-def85ff5750f admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:55.125219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a07ab-7353-4e48-aaf0-d34da39b8067 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:55.127417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a07ab-7353-4e48-aaf0-d34da39b8067 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:55.127417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a07ab-7353-4e48-aaf0-d34da39b8067 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:55.127417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a07ab-7353-4e48-aaf0-d34da39b8067 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:55.137415 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a07ab-7353-4e48-aaf0-d34da39b8067 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:55.139153 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a07ab-7353-4e48-aaf0-d34da39b8067 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:55.150438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1229/2467] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:40:55 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:40:55.342134 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008d54e-661b-4e9f-a6ad-d5374d7ec7b5 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:55.444281 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008d54e-661b-4e9f-a6ad-d5374d7ec7b5 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:55.452958 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849c64b-a725-4de1-b75c-9699a1ba1456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:55.456559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849c64b-a725-4de1-b75c-9699a1ba1456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:55.456559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849c64b-a725-4de1-b75c-9699a1ba1456 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:55.456559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849c64b-a725-4de1-b75c-9699a1ba1456 admin admin] PATH_INFO: `/v3/users/4a2044bcebbf47e791dc004f2ed2800d` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:55.456559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849c64b-a725-4de1-b75c-9699a1ba1456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a2044bcebbf47e791dc004f2ed2800d)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:55.459665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849c64b-a725-4de1-b75c-9699a1ba1456 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:55.710042 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c849c64b-a725-4de1-b75c-9699a1ba1456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a2044bcebbf47e791dc004f2ed2800d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:55.826296 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c849c64b-a725-4de1-b75c-9699a1ba1456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a2044bcebbf47e791dc004f2ed2800d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:55.827376 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c849c64b-a725-4de1-b75c-9699a1ba1456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a2044bcebbf47e791dc004f2ed2800d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:55.828577 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c849c64b-a725-4de1-b75c-9699a1ba1456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a2044bcebbf47e791dc004f2ed2800d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:55.844651 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-c849c64b-a725-4de1-b75c-9699a1ba1456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a2044bcebbf47e791dc004f2ed2800d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:55.884382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1239/2468] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:40:55 2022] DELETE /identity/v3/users/4a2044bcebbf47e791dc004f2ed2800d => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:55.891313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e80b6-b136-460a-915a-d895450faa67 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:55.892237 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e80b6-b136-460a-915a-d895450faa67 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:55.892455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e80b6-b136-460a-915a-d895450faa67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:55.928099 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-a73e80b6-b136-460a-915a-d895450faa67 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:55.930328 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a73e80b6-b136-460a-915a-d895450faa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:56.043864 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1230/2469] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:56.057807 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849c64b-a725-4de1-b75c-9699a1ba1456 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:56.061517 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c849c64b-a725-4de1-b75c-9699a1ba1456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:56.153849 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849c64b-a725-4de1-b75c-9699a1ba1456 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:56.159028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07a42e-4143-40b0-9359-dd818e036bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:56.165860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07a42e-4143-40b0-9359-dd818e036bbd service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:56.165860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07a42e-4143-40b0-9359-dd818e036bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:56.165860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07a42e-4143-40b0-9359-dd818e036bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:56.173882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07a42e-4143-40b0-9359-dd818e036bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:56.176373 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07a42e-4143-40b0-9359-dd818e036bbd service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:56.190624 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1240/2470] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:40:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:40:56.853117 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e80b6-b136-460a-915a-d895450faa67 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:56.855639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a73e80b6-b136-460a-915a-d895450faa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:56.870519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e80b6-b136-460a-915a-d895450faa67 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:56.877609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0de877-dfee-4a7f-8e82-d8f353c45b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:56.878387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0de877-dfee-4a7f-8e82-d8f353c45b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:56.878675 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0de877-dfee-4a7f-8e82-d8f353c45b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:56.878945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0de877-dfee-4a7f-8e82-d8f353c45b79 admin admin] PATH_INFO: `/v3/projects/e019ccc8d69c4238940e7d3a7c3ce5d3` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:56.880481 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0de877-dfee-4a7f-8e82-d8f353c45b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e019ccc8d69c4238940e7d3a7c3ce5d3)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:56.882958 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0de877-dfee-4a7f-8e82-d8f353c45b79 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:57.028342 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:57.051041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-cb0de877-dfee-4a7f-8e82-d8f353c45b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e019ccc8d69c4238940e7d3a7c3ce5d3', 'request_id': 'req-cb0de877-dfee-4a7f-8e82-d8f353c45b79'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:57.061082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-cb0de877-dfee-4a7f-8e82-d8f353c45b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e019ccc8d69c4238940e7d3a7c3ce5d3', 'request_id': 'req-cb0de877-dfee-4a7f-8e82-d8f353c45b79'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:57.169528 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-cb0de877-dfee-4a7f-8e82-d8f353c45b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e019ccc8d69c4238940e7d3a7c3ce5d3', 'request_id': 'req-cb0de877-dfee-4a7f-8e82-d8f353c45b79'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:57.187786 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-cb0de877-dfee-4a7f-8e82-d8f353c45b79 admin admin] The token cache is being invalidate because project e019ccc8d69c4238940e7d3a7c3ce5d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:40:57.188239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-cb0de877-dfee-4a7f-8e82-d8f353c45b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:57.189094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-cb0de877-dfee-4a7f-8e82-d8f353c45b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:57.191047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1231/2471] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:40:56 2022] DELETE /identity/v3/projects/e019ccc8d69c4238940e7d3a7c3ce5d3 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:57.196027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07a42e-4143-40b0-9359-dd818e036bbd service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:57.208781 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07a42e-4143-40b0-9359-dd818e036bbd service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:57.229809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:57.290947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb3abe-651f-4c6c-9536-405fd10ed0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:57.291713 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb3abe-651f-4c6c-9536-405fd10ed0fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:57.292024 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb3abe-651f-4c6c-9536-405fd10ed0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:57.292309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb3abe-651f-4c6c-9536-405fd10ed0fa admin admin] PATH_INFO: `/v3/users/5962c6330fe84206b0408f1414124c2a` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:57.295423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb3abe-651f-4c6c-9536-405fd10ed0fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=5962c6330fe84206b0408f1414124c2a)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:57.297760 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb3abe-651f-4c6c-9536-405fd10ed0fa admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:57.690584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2ecb3abe-651f-4c6c-9536-405fd10ed0fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5962c6330fe84206b0408f1414124c2a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:57.864799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2ecb3abe-651f-4c6c-9536-405fd10ed0fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5962c6330fe84206b0408f1414124c2a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:57.865507 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2ecb3abe-651f-4c6c-9536-405fd10ed0fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5962c6330fe84206b0408f1414124c2a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:57.866177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2ecb3abe-651f-4c6c-9536-405fd10ed0fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5962c6330fe84206b0408f1414124c2a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:57.878512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2ecb3abe-651f-4c6c-9536-405fd10ed0fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5962c6330fe84206b0408f1414124c2a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:57.909838 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1241/2472] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:40:57 2022] DELETE /identity/v3/users/5962c6330fe84206b0408f1414124c2a => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:58.457262 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0de877-dfee-4a7f-8e82-d8f353c45b79 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:58.462233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-cb0de877-dfee-4a7f-8e82-d8f353c45b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:58.483145 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0de877-dfee-4a7f-8e82-d8f353c45b79 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:58.563892 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a9ad0-3944-4bf4-bd8d-442f7b28d47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:58.564619 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a9ad0-3944-4bf4-bd8d-442f7b28d47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:58.565006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a9ad0-3944-4bf4-bd8d-442f7b28d47a admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:58.565389 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a9ad0-3944-4bf4-bd8d-442f7b28d47a admin admin] PATH_INFO: `/v3/projects/40cf13ab45994aba996fae12564c3c63` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:58.567112 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a9ad0-3944-4bf4-bd8d-442f7b28d47a admin admin] RBAC: Authorizing `identity:delete_project(project_id=40cf13ab45994aba996fae12564c3c63)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:58.569323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a9ad0-3944-4bf4-bd8d-442f7b28d47a admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:58.718462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3d6a9ad0-3944-4bf4-bd8d-442f7b28d47a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40cf13ab45994aba996fae12564c3c63', 'request_id': 'req-3d6a9ad0-3944-4bf4-bd8d-442f7b28d47a'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:58.726600 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3d6a9ad0-3944-4bf4-bd8d-442f7b28d47a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40cf13ab45994aba996fae12564c3c63', 'request_id': 'req-3d6a9ad0-3944-4bf4-bd8d-442f7b28d47a'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:58.823024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3d6a9ad0-3944-4bf4-bd8d-442f7b28d47a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40cf13ab45994aba996fae12564c3c63', 'request_id': 'req-3d6a9ad0-3944-4bf4-bd8d-442f7b28d47a'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:58.840706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3d6a9ad0-3944-4bf4-bd8d-442f7b28d47a admin admin] The token cache is being invalidate because project 40cf13ab45994aba996fae12564c3c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:40:58.841053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3d6a9ad0-3944-4bf4-bd8d-442f7b28d47a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:58.842068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3d6a9ad0-3944-4bf4-bd8d-442f7b28d47a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:40:58.844680 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1232/2473] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:40:58 2022] DELETE /identity/v3/projects/40cf13ab45994aba996fae12564c3c63 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:40:58.852571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78fb46-1587-4c0f-8097-b1eba2465f0d None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:58.852571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78fb46-1587-4c0f-8097-b1eba2465f0d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:58.853501 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78fb46-1587-4c0f-8097-b1eba2465f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:58.880118 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-4f78fb46-1587-4c0f-8097-b1eba2465f0d None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:58.881403 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-4f78fb46-1587-4c0f-8097-b1eba2465f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:58.966388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1242/2474] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:40:58.975432 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403852c4-33cd-456d-b61e-0b3d4e45014c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:58.975722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403852c4-33cd-456d-b61e-0b3d4e45014c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:58.976038 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403852c4-33cd-456d-b61e-0b3d4e45014c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:59.007269 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-403852c4-33cd-456d-b61e-0b3d4e45014c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:59.009492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-403852c4-33cd-456d-b61e-0b3d4e45014c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:59.025172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1233/2475] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:40:59.039746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78fb46-1587-4c0f-8097-b1eba2465f0d None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:59.050775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78fb46-1587-4c0f-8097-b1eba2465f0d None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:59.056926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6625e-7563-4c08-9aff-a6cbae2be7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:59.058628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6625e-7563-4c08-9aff-a6cbae2be7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:59.058952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6625e-7563-4c08-9aff-a6cbae2be7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:59.059304 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6625e-7563-4c08-9aff-a6cbae2be7b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:59.060156 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6625e-7563-4c08-9aff-a6cbae2be7b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:40:59.060526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6625e-7563-4c08-9aff-a6cbae2be7b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:59.062225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6625e-7563-4c08-9aff-a6cbae2be7b5 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:59.077785 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1243/2476] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:40:59.082608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403852c4-33cd-456d-b61e-0b3d4e45014c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:59.094850 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403852c4-33cd-456d-b61e-0b3d4e45014c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:59.099103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92108b60-7761-4a26-a9b4-e672fdcde649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:59.099729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92108b60-7761-4a26-a9b4-e672fdcde649 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:59.100014 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92108b60-7761-4a26-a9b4-e672fdcde649 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:59.100295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92108b60-7761-4a26-a9b4-e672fdcde649 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:59.100943 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92108b60-7761-4a26-a9b4-e672fdcde649 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:59.102864 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92108b60-7761-4a26-a9b4-e672fdcde649 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:59.275007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1234/2477] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:40:59 2022] POST /identity/v3/projects => generated 363 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:40:59.280691 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6625e-7563-4c08-9aff-a6cbae2be7b5 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:59.292121 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6625e-7563-4c08-9aff-a6cbae2be7b5 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:59.335492 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:59.392190 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d0ba5-fbcf-4c15-924c-6828ee32b9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:59.392992 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d0ba5-fbcf-4c15-924c-6828ee32b9a8 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:59.393307 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d0ba5-fbcf-4c15-924c-6828ee32b9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:59.393581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d0ba5-fbcf-4c15-924c-6828ee32b9a8 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:59.394284 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d0ba5-fbcf-4c15-924c-6828ee32b9a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:59.396332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d0ba5-fbcf-4c15-924c-6828ee32b9a8 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:59.556394 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1244/2478] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:40:59 2022] POST /identity/v3/users => generated 282 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:40:59.563926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92108b60-7761-4a26-a9b4-e672fdcde649 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:59.575791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92108b60-7761-4a26-a9b4-e672fdcde649 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:59.583685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac72bc-0b41-41e9-af07-a248df70e4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:59.583685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac72bc-0b41-41e9-af07-a248df70e4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:59.584060 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac72bc-0b41-41e9-af07-a248df70e4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:59.584281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac72bc-0b41-41e9-af07-a248df70e4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:59.584950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac72bc-0b41-41e9-af07-a248df70e4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:59.586596 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac72bc-0b41-41e9-af07-a248df70e4d3 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:59.600151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1235/2479] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:40:59.605019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d0ba5-fbcf-4c15-924c-6828ee32b9a8 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:59.617601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d0ba5-fbcf-4c15-924c-6828ee32b9a8 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:59.623370 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa9c6b-c63c-4da3-a70d-0ba0c278a012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:59.624010 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa9c6b-c63c-4da3-a70d-0ba0c278a012 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:59.624291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa9c6b-c63c-4da3-a70d-0ba0c278a012 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:59.628809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa9c6b-c63c-4da3-a70d-0ba0c278a012 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:59.628809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa9c6b-c63c-4da3-a70d-0ba0c278a012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:59.628809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa9c6b-c63c-4da3-a70d-0ba0c278a012 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:59.641697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1245/2480] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:40:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:40:59.647633 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac72bc-0b41-41e9-af07-a248df70e4d3 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:40:59.654899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:40:59.659823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac72bc-0b41-41e9-af07-a248df70e4d3 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:40:59.665671 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a6956-40af-49cd-a8f8-8e6b93ad8516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:40:59.666463 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6956-40af-49cd-a8f8-8e6b93ad8516 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:59.666745 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6956-40af-49cd-a8f8-8e6b93ad8516 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:59.667290 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6956-40af-49cd-a8f8-8e6b93ad8516 admin admin] PATH_INFO: `/v3/projects/d27d7514f1384589a91816bc0cbd8c1a/users/ecc8d9701e5349f1a34e7ceae386384e/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:59.687882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a6956-40af-49cd-a8f8-8e6b93ad8516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d27d7514f1384589a91816bc0cbd8c1a, user_id=ecc8d9701e5349f1a34e7ceae386384e, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:40:59.696519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a6956-40af-49cd-a8f8-8e6b93ad8516 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:40:59.781613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1236/2481] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:40:59 2022] PUT /identity/v3/projects/d27d7514f1384589a91816bc0cbd8c1a/users/ecc8d9701e5349f1a34e7ceae386384e/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:40:59.794120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac33c83-05a2-466e-a218-de3d63283669 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:40:59.794590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac33c83-05a2-466e-a218-de3d63283669 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:40:59.794779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac33c83-05a2-466e-a218-de3d63283669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:40:59.830719 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-eac33c83-05a2-466e-a218-de3d63283669 None None] MFA Rules not processed for user `ecc8d9701e5349f1a34e7ceae386384e`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:40:59.832099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-eac33c83-05a2-466e-a218-de3d63283669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:40:59.970222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1246/2482] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:40:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:40:59.975732 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a6956-40af-49cd-a8f8-8e6b93ad8516 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:00.073515 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a6956-40af-49cd-a8f8-8e6b93ad8516 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:00.082716 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9964a5-78c9-4efd-8fe1-90d7b1ab1e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:00.083404 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9964a5-78c9-4efd-8fe1-90d7b1ab1e02 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:00.083684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9964a5-78c9-4efd-8fe1-90d7b1ab1e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:00.084052 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9964a5-78c9-4efd-8fe1-90d7b1ab1e02 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:00.085548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9964a5-78c9-4efd-8fe1-90d7b1ab1e02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:00.087916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9964a5-78c9-4efd-8fe1-90d7b1ab1e02 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:00.218812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1237/2483] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:40:59 2022] POST /identity/v3/projects => generated 365 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:41:00.225060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac33c83-05a2-466e-a218-de3d63283669 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:00.243781 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac33c83-05a2-466e-a218-de3d63283669 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:00.366618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e0f2e-1c0d-4e88-b5d9-ce51b34dfa1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:00.366618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e0f2e-1c0d-4e88-b5d9-ce51b34dfa1a admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:00.366618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e0f2e-1c0d-4e88-b5d9-ce51b34dfa1a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:00.366618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e0f2e-1c0d-4e88-b5d9-ce51b34dfa1a admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:00.367759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e0f2e-1c0d-4e88-b5d9-ce51b34dfa1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:00.371879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e0f2e-1c0d-4e88-b5d9-ce51b34dfa1a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:00.532175 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1247/2484] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:41:00 2022] POST /identity/v3/users => generated 289 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:41:00.537938 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9964a5-78c9-4efd-8fe1-90d7b1ab1e02 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:00.556486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9964a5-78c9-4efd-8fe1-90d7b1ab1e02 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:00.564552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2475e-07dd-4e32-a806-4ca68cf0b286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:00.565615 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2475e-07dd-4e32-a806-4ca68cf0b286 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:00.566117 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2475e-07dd-4e32-a806-4ca68cf0b286 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:00.566607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2475e-07dd-4e32-a806-4ca68cf0b286 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:00.567926 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2475e-07dd-4e32-a806-4ca68cf0b286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:00.572375 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2475e-07dd-4e32-a806-4ca68cf0b286 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:00.594621 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1238/2485] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:41:00.599884 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e0f2e-1c0d-4e88-b5d9-ce51b34dfa1a admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:00.613274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e0f2e-1c0d-4e88-b5d9-ce51b34dfa1a admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:00.619109 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55354d9b-5295-4d23-90c3-bc86ac57a1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:00.619887 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55354d9b-5295-4d23-90c3-bc86ac57a1c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:00.623927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55354d9b-5295-4d23-90c3-bc86ac57a1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:00.623927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55354d9b-5295-4d23-90c3-bc86ac57a1c5 admin admin] PATH_INFO: `/v3/projects/8147f57388d04bf4b1e82c3817b9db10/users/fbcd04b1fcc948a7a01c91692a7664be/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:00.640573 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55354d9b-5295-4d23-90c3-bc86ac57a1c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8147f57388d04bf4b1e82c3817b9db10, user_id=fbcd04b1fcc948a7a01c91692a7664be, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:00.643935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55354d9b-5295-4d23-90c3-bc86ac57a1c5 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:00.738627 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:00.751234 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1248/2486] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:41:00 2022] PUT /identity/v3/projects/8147f57388d04bf4b1e82c3817b9db10/users/fbcd04b1fcc948a7a01c91692a7664be/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:00.757059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2475e-07dd-4e32-a806-4ca68cf0b286 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:00.772304 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2475e-07dd-4e32-a806-4ca68cf0b286 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:00.874028 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a37638-395e-44ce-986f-096e0fdfd18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:00.875260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a37638-395e-44ce-986f-096e0fdfd18d admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:00.875695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a37638-395e-44ce-986f-096e0fdfd18d admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:00.877586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a37638-395e-44ce-986f-096e0fdfd18d admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:00.877586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a37638-395e-44ce-986f-096e0fdfd18d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:00.880630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a37638-395e-44ce-986f-096e0fdfd18d admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:00.903238 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1239/2487] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:41:00.909133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55354d9b-5295-4d23-90c3-bc86ac57a1c5 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:00.928050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55354d9b-5295-4d23-90c3-bc86ac57a1c5 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:00.936072 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3814d53-c366-4475-ad43-e432013c47f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:00.937428 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3814d53-c366-4475-ad43-e432013c47f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:00.937931 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3814d53-c366-4475-ad43-e432013c47f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:00.938464 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3814d53-c366-4475-ad43-e432013c47f6 admin admin] PATH_INFO: `/v3/projects/8147f57388d04bf4b1e82c3817b9db10/users/fbcd04b1fcc948a7a01c91692a7664be/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:00.941374 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3814d53-c366-4475-ad43-e432013c47f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8147f57388d04bf4b1e82c3817b9db10, user_id=fbcd04b1fcc948a7a01c91692a7664be, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:00.944704 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3814d53-c366-4475-ad43-e432013c47f6 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:00.960865 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1249/2488] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:41:00 2022] PUT /identity/v3/projects/8147f57388d04bf4b1e82c3817b9db10/users/fbcd04b1fcc948a7a01c91692a7664be/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:00.982856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d999f4-8741-469c-b617-d5f79e4eb421 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:00.982856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d999f4-8741-469c-b617-d5f79e4eb421 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:00.982856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d999f4-8741-469c-b617-d5f79e4eb421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:01.033851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-d2d999f4-8741-469c-b617-d5f79e4eb421 None None] MFA Rules not processed for user `fbcd04b1fcc948a7a01c91692a7664be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:01.036368 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d2d999f4-8741-469c-b617-d5f79e4eb421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:01.097146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:01.134558 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1240/2489] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:01.150824 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3814d53-c366-4475-ad43-e432013c47f6 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:01.156837 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f3814d53-c366-4475-ad43-e432013c47f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:01.240846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3814d53-c366-4475-ad43-e432013c47f6 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:01.244896 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e2abf-c945-475c-a835-26a6c8955f66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:01.245530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e2abf-c945-475c-a835-26a6c8955f66 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:01.246526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e2abf-c945-475c-a835-26a6c8955f66 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:01.246800 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e2abf-c945-475c-a835-26a6c8955f66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:01.257564 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e2abf-c945-475c-a835-26a6c8955f66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:01.259168 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e2abf-c945-475c-a835-26a6c8955f66 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:01.275593 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1250/2490] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:41:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:41:01.633915 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d999f4-8741-469c-b617-d5f79e4eb421 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:01.648395 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d999f4-8741-469c-b617-d5f79e4eb421 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:01.653210 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe60e7-e947-45af-8cf7-b8f9bc556b5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:01.654312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe60e7-e947-45af-8cf7-b8f9bc556b5b service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:01.654643 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe60e7-e947-45af-8cf7-b8f9bc556b5b service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:01.654917 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe60e7-e947-45af-8cf7-b8f9bc556b5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:01.744978 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe60e7-e947-45af-8cf7-b8f9bc556b5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:01.747267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe60e7-e947-45af-8cf7-b8f9bc556b5b service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:01.758183 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1241/2491] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:41:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:41:03.164140 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e2abf-c945-475c-a835-26a6c8955f66 service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:03.165905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-776e2abf-c945-475c-a835-26a6c8955f66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:03.242862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e2abf-c945-475c-a835-26a6c8955f66 service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:03.248387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f01c8-4a00-4be1-9ef9-c362ca80814d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:03.251081 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f01c8-4a00-4be1-9ef9-c362ca80814d service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:03.251081 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f01c8-4a00-4be1-9ef9-c362ca80814d service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:03.251081 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f01c8-4a00-4be1-9ef9-c362ca80814d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:03.252028 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b56f01c8-4a00-4be1-9ef9-c362ca80814d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:03.375110 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f01c8-4a00-4be1-9ef9-c362ca80814d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:03.376974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f01c8-4a00-4be1-9ef9-c362ca80814d service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:03.391189 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1251/2492] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:41:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:41:05.154101 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe60e7-e947-45af-8cf7-b8f9bc556b5b service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:05.156201 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-bdbe60e7-e947-45af-8cf7-b8f9bc556b5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:05.186919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe60e7-e947-45af-8cf7-b8f9bc556b5b service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:05.192992 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c967de-74fa-478a-b6c2-f3be7233420b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:05.194173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c967de-74fa-478a-b6c2-f3be7233420b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:05.194460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c967de-74fa-478a-b6c2-f3be7233420b admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:05.195226 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c967de-74fa-478a-b6c2-f3be7233420b admin admin] PATH_INFO: `/v3/users/4c51481b58254cf49831a865f3d46579` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:05.196985 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c967de-74fa-478a-b6c2-f3be7233420b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c51481b58254cf49831a865f3d46579)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:05.200911 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c967de-74fa-478a-b6c2-f3be7233420b admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:05.437752 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-42c967de-74fa-478a-b6c2-f3be7233420b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c51481b58254cf49831a865f3d46579'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:05.481759 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-42c967de-74fa-478a-b6c2-f3be7233420b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c51481b58254cf49831a865f3d46579'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:05.487127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-42c967de-74fa-478a-b6c2-f3be7233420b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c51481b58254cf49831a865f3d46579'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:05.496999 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-42c967de-74fa-478a-b6c2-f3be7233420b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c51481b58254cf49831a865f3d46579'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:05.499221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-42c967de-74fa-478a-b6c2-f3be7233420b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c51481b58254cf49831a865f3d46579'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:05.663297 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1242/2493] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:41:05 2022] DELETE /identity/v3/users/4c51481b58254cf49831a865f3d46579 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:05.670079 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c9c20-323b-47c6-8f61-a34019c7c8f9 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:05.670693 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c9c20-323b-47c6-8f61-a34019c7c8f9 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:05.670693 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c9c20-323b-47c6-8f61-a34019c7c8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:05.700103 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-449c9c20-323b-47c6-8f61-a34019c7c8f9 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:05.702263 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-449c9c20-323b-47c6-8f61-a34019c7c8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:05.796219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:05.872274 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1252/2494] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:05.896719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c967de-74fa-478a-b6c2-f3be7233420b admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:05.903720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-42c967de-74fa-478a-b6c2-f3be7233420b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:06.030621 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:06.043345 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c967de-74fa-478a-b6c2-f3be7233420b admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:06.054868 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c374023-a261-4b97-a436-5a74244fd99f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:06.057502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c374023-a261-4b97-a436-5a74244fd99f service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:06.057502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c374023-a261-4b97-a436-5a74244fd99f service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:06.057854 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c374023-a261-4b97-a436-5a74244fd99f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:06.075143 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c374023-a261-4b97-a436-5a74244fd99f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:06.079016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c374023-a261-4b97-a436-5a74244fd99f service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:06.105183 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1243/2495] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:41:07.197187 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c9c20-323b-47c6-8f61-a34019c7c8f9 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:07.201387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-449c9c20-323b-47c6-8f61-a34019c7c8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:07.248292 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c9c20-323b-47c6-8f61-a34019c7c8f9 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:07.260291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe5104-43d0-4474-a896-bcc69882f959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:07.262511 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe5104-43d0-4474-a896-bcc69882f959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:07.263046 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe5104-43d0-4474-a896-bcc69882f959 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:07.264562 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe5104-43d0-4474-a896-bcc69882f959 admin admin] PATH_INFO: `/v3/projects/0af1f942d50647648e712882c3bb7909` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:07.266714 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe5104-43d0-4474-a896-bcc69882f959 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0af1f942d50647648e712882c3bb7909)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:07.270795 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe5104-43d0-4474-a896-bcc69882f959 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:07.578582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-34fe5104-43d0-4474-a896-bcc69882f959 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0af1f942d50647648e712882c3bb7909', 'request_id': 'req-34fe5104-43d0-4474-a896-bcc69882f959'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:07.596500 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-34fe5104-43d0-4474-a896-bcc69882f959 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0af1f942d50647648e712882c3bb7909', 'request_id': 'req-34fe5104-43d0-4474-a896-bcc69882f959'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:07.745101 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-34fe5104-43d0-4474-a896-bcc69882f959 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0af1f942d50647648e712882c3bb7909', 'request_id': 'req-34fe5104-43d0-4474-a896-bcc69882f959'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:07.764700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-34fe5104-43d0-4474-a896-bcc69882f959 admin admin] The token cache is being invalidate because project 0af1f942d50647648e712882c3bb7909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:41:07.765304 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-34fe5104-43d0-4474-a896-bcc69882f959 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:07.766533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-34fe5104-43d0-4474-a896-bcc69882f959 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:07.768766 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1253/2496] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:41:07 2022] DELETE /identity/v3/projects/0af1f942d50647648e712882c3bb7909 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:41:07.775070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601fbaf-3906-475d-b4d0-41f1a9adee88 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:07.777430 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601fbaf-3906-475d-b4d0-41f1a9adee88 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:07.778239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601fbaf-3906-475d-b4d0-41f1a9adee88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:07.818490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-6601fbaf-3906-475d-b4d0-41f1a9adee88 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:07.821300 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6601fbaf-3906-475d-b4d0-41f1a9adee88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:07.988265 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1244/2497] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:07.999154 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723afc0-3433-4d0f-83ed-2e1e5551fe0d None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:07.999448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723afc0-3433-4d0f-83ed-2e1e5551fe0d None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:07.999754 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723afc0-3433-4d0f-83ed-2e1e5551fe0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:08.035050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-1723afc0-3433-4d0f-83ed-2e1e5551fe0d None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:08.036535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1723afc0-3433-4d0f-83ed-2e1e5551fe0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:08.047757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1254/2498] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:08.053191 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601fbaf-3906-475d-b4d0-41f1a9adee88 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:08.084587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601fbaf-3906-475d-b4d0-41f1a9adee88 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:08.112780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b65df-a46b-45e5-bed0-f47c82b23c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:08.112780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b65df-a46b-45e5-bed0-f47c82b23c3a admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:08.112780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b65df-a46b-45e5-bed0-f47c82b23c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:08.112780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b65df-a46b-45e5-bed0-f47c82b23c3a admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:08.113363 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b65df-a46b-45e5-bed0-f47c82b23c3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:41:08.113363 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b65df-a46b-45e5-bed0-f47c82b23c3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:08.115700 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b65df-a46b-45e5-bed0-f47c82b23c3a admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:08.150153 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1245/2499] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:41:08.156831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723afc0-3433-4d0f-83ed-2e1e5551fe0d None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:08.173445 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723afc0-3433-4d0f-83ed-2e1e5551fe0d None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:08.187451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbc7d9-e4d1-4d58-b9e8-e81c37062b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:08.189639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbc7d9-e4d1-4d58-b9e8-e81c37062b56 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:08.190635 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbc7d9-e4d1-4d58-b9e8-e81c37062b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:08.192385 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbc7d9-e4d1-4d58-b9e8-e81c37062b56 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:08.194602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbc7d9-e4d1-4d58-b9e8-e81c37062b56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:08.201223 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbc7d9-e4d1-4d58-b9e8-e81c37062b56 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:08.377746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1255/2500] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:41:08 2022] POST /identity/v3/projects => generated 365 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:41:08.382997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b65df-a46b-45e5-bed0-f47c82b23c3a admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:08.396273 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b65df-a46b-45e5-bed0-f47c82b23c3a admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:08.517158 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840b3e0-ccc2-4125-b30a-ab663516dd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:08.518105 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840b3e0-ccc2-4125-b30a-ab663516dd76 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:08.518599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840b3e0-ccc2-4125-b30a-ab663516dd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:08.519119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840b3e0-ccc2-4125-b30a-ab663516dd76 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:08.520398 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840b3e0-ccc2-4125-b30a-ab663516dd76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:08.524229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840b3e0-ccc2-4125-b30a-ab663516dd76 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:08.654137 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1246/2501] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:41:08 2022] POST /identity/v3/users => generated 283 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:41:08.660423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbc7d9-e4d1-4d58-b9e8-e81c37062b56 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:08.678102 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbc7d9-e4d1-4d58-b9e8-e81c37062b56 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:08.688534 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af864a64-859a-4084-83de-e6b6c1da1d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:08.689904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af864a64-859a-4084-83de-e6b6c1da1d3d admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:08.690598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af864a64-859a-4084-83de-e6b6c1da1d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:08.691166 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af864a64-859a-4084-83de-e6b6c1da1d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:08.692589 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af864a64-859a-4084-83de-e6b6c1da1d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:08.695164 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af864a64-859a-4084-83de-e6b6c1da1d3d admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:08.713100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1256/2502] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:41:08.719136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840b3e0-ccc2-4125-b30a-ab663516dd76 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:08.743924 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840b3e0-ccc2-4125-b30a-ab663516dd76 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:08.752978 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474a284-1ff2-4bbe-9b69-10c09d7177fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:08.760094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474a284-1ff2-4bbe-9b69-10c09d7177fa admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:08.760730 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474a284-1ff2-4bbe-9b69-10c09d7177fa admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:08.763306 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474a284-1ff2-4bbe-9b69-10c09d7177fa admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:08.763306 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474a284-1ff2-4bbe-9b69-10c09d7177fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:08.765590 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474a284-1ff2-4bbe-9b69-10c09d7177fa admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:08.791115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1247/2503] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:41:08.793102 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af864a64-859a-4084-83de-e6b6c1da1d3d admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:08.812072 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af864a64-859a-4084-83de-e6b6c1da1d3d admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:08.820892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216dea2-76bc-4ed0-acb4-ff7fa73267e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:08.822054 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216dea2-76bc-4ed0-acb4-ff7fa73267e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:08.822523 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216dea2-76bc-4ed0-acb4-ff7fa73267e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:08.823123 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216dea2-76bc-4ed0-acb4-ff7fa73267e3 admin admin] PATH_INFO: `/v3/projects/e03f4c40f3d94baba4adc3e9e250b4e3/users/cab8ca33d0f040358d15eb24c8ee5c4b/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:08.853902 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216dea2-76bc-4ed0-acb4-ff7fa73267e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e03f4c40f3d94baba4adc3e9e250b4e3, user_id=cab8ca33d0f040358d15eb24c8ee5c4b, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:08.859154 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216dea2-76bc-4ed0-acb4-ff7fa73267e3 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:09.001184 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1257/2504] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:41:08 2022] PUT /identity/v3/projects/e03f4c40f3d94baba4adc3e9e250b4e3/users/cab8ca33d0f040358d15eb24c8ee5c4b/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:09.021879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb098eb0-dd79-4be7-87cc-b3b49a499240 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:09.022303 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb098eb0-dd79-4be7-87cc-b3b49a499240 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:09.022724 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb098eb0-dd79-4be7-87cc-b3b49a499240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:09.084386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-eb098eb0-dd79-4be7-87cc-b3b49a499240 None None] MFA Rules not processed for user `cab8ca33d0f040358d15eb24c8ee5c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:09.086757 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-eb098eb0-dd79-4be7-87cc-b3b49a499240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:09.179602 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:09.213880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1248/2505] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:09.235143 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216dea2-76bc-4ed0-acb4-ff7fa73267e3 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:09.242228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-2216dea2-76bc-4ed0-acb4-ff7fa73267e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:09.365688 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216dea2-76bc-4ed0-acb4-ff7fa73267e3 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:09.374528 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ec3ed-3451-4ad7-8dbf-92303510ead6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:09.375733 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ec3ed-3451-4ad7-8dbf-92303510ead6 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:09.376221 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ec3ed-3451-4ad7-8dbf-92303510ead6 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:09.376710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ec3ed-3451-4ad7-8dbf-92303510ead6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:09.398148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ec3ed-3451-4ad7-8dbf-92303510ead6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:09.400891 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ec3ed-3451-4ad7-8dbf-92303510ead6 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:09.420170 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1258/2506] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:41:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:41:10.799684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb098eb0-dd79-4be7-87cc-b3b49a499240 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:10.801314 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-eb098eb0-dd79-4be7-87cc-b3b49a499240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:10.899443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb098eb0-dd79-4be7-87cc-b3b49a499240 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:10.907041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74499ff9-d0c8-4ac0-ab06-45b930e4119f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:10.907877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74499ff9-d0c8-4ac0-ab06-45b930e4119f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:10.908340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74499ff9-d0c8-4ac0-ab06-45b930e4119f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:10.908656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74499ff9-d0c8-4ac0-ab06-45b930e4119f admin admin] PATH_INFO: `/v3/users/c5e931aa22344990a0f032159ebc20ad` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:10.910457 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74499ff9-d0c8-4ac0-ab06-45b930e4119f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5e931aa22344990a0f032159ebc20ad)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:10.912513 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74499ff9-d0c8-4ac0-ab06-45b930e4119f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:11.310681 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-74499ff9-d0c8-4ac0-ab06-45b930e4119f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5e931aa22344990a0f032159ebc20ad'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:11.328552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-74499ff9-d0c8-4ac0-ab06-45b930e4119f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5e931aa22344990a0f032159ebc20ad'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:11.329220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-74499ff9-d0c8-4ac0-ab06-45b930e4119f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5e931aa22344990a0f032159ebc20ad'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:11.338482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-74499ff9-d0c8-4ac0-ab06-45b930e4119f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5e931aa22344990a0f032159ebc20ad'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:11.339927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-74499ff9-d0c8-4ac0-ab06-45b930e4119f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5e931aa22344990a0f032159ebc20ad'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:11.557186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1249/2507] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:41:10 2022] DELETE /identity/v3/users/c5e931aa22344990a0f032159ebc20ad => generated 0 bytes in 758 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:41:12.213663 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ec3ed-3451-4ad7-8dbf-92303510ead6 service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:12.224590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:12.234201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ec3ed-3451-4ad7-8dbf-92303510ead6 service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:12.338558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78f525-3474-41b5-9900-1a5b61066527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:12.340275 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78f525-3474-41b5-9900-1a5b61066527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:12.340275 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78f525-3474-41b5-9900-1a5b61066527 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:12.340749 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78f525-3474-41b5-9900-1a5b61066527 admin admin] PATH_INFO: `/v3/projects/495aadbbe15c4f18a5018f45e8bab423` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:12.343268 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca78f525-3474-41b5-9900-1a5b61066527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=495aadbbe15c4f18a5018f45e8bab423)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:12.347940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca78f525-3474-41b5-9900-1a5b61066527 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:12.546408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-ca78f525-3474-41b5-9900-1a5b61066527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '495aadbbe15c4f18a5018f45e8bab423', 'request_id': 'req-ca78f525-3474-41b5-9900-1a5b61066527'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:12.555674 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-ca78f525-3474-41b5-9900-1a5b61066527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '495aadbbe15c4f18a5018f45e8bab423', 'request_id': 'req-ca78f525-3474-41b5-9900-1a5b61066527'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:12.672027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-ca78f525-3474-41b5-9900-1a5b61066527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '495aadbbe15c4f18a5018f45e8bab423', 'request_id': 'req-ca78f525-3474-41b5-9900-1a5b61066527'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:12.688998 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-ca78f525-3474-41b5-9900-1a5b61066527 admin admin] The token cache is being invalidate because project 495aadbbe15c4f18a5018f45e8bab423 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:41:12.689309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-ca78f525-3474-41b5-9900-1a5b61066527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:12.690460 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-ca78f525-3474-41b5-9900-1a5b61066527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:12.694258 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1259/2508] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:41:12 2022] DELETE /identity/v3/projects/495aadbbe15c4f18a5018f45e8bab423 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:12.700795 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3d294-4f8f-4074-a653-7ed483136302 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:12.701598 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3d294-4f8f-4074-a653-7ed483136302 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:12.701598 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3d294-4f8f-4074-a653-7ed483136302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:12.727553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-ffc3d294-4f8f-4074-a653-7ed483136302 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:12.728884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ffc3d294-4f8f-4074-a653-7ed483136302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:12.830905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1250/2509] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:41:12.843366 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d12a-f511-43d7-bdc1-efa81821e67f None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:12.843768 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d12a-f511-43d7-bdc1-efa81821e67f None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:12.844088 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d12a-f511-43d7-bdc1-efa81821e67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:12.876571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-d0c6d12a-f511-43d7-bdc1-efa81821e67f None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:12.879389 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-d0c6d12a-f511-43d7-bdc1-efa81821e67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:12.896783 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1260/2510] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:12.898779 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3d294-4f8f-4074-a653-7ed483136302 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:12.911966 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:12.919054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3d294-4f8f-4074-a653-7ed483136302 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:12.926199 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587297c-100e-404c-88a5-724c0787d145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:12.927256 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587297c-100e-404c-88a5-724c0787d145 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:12.927715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587297c-100e-404c-88a5-724c0787d145 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:12.928183 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587297c-100e-404c-88a5-724c0787d145 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:12.929333 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587297c-100e-404c-88a5-724c0787d145 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:41:12.929892 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587297c-100e-404c-88a5-724c0787d145 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:12.933497 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587297c-100e-404c-88a5-724c0787d145 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:12.956711 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1251/2511] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:41:12.961615 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6d12a-f511-43d7-bdc1-efa81821e67f None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:12.978882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6d12a-f511-43d7-bdc1-efa81821e67f None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:12.985364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ffcb7-6693-4bc5-be9e-b52917e99939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:12.986020 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ffcb7-6693-4bc5-be9e-b52917e99939 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:12.986292 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ffcb7-6693-4bc5-be9e-b52917e99939 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:12.987286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ffcb7-6693-4bc5-be9e-b52917e99939 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:12.987286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ffcb7-6693-4bc5-be9e-b52917e99939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:12.989276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ffcb7-6693-4bc5-be9e-b52917e99939 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:13.133596 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1261/2512] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:41:12 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:41:13.135224 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587297c-100e-404c-88a5-724c0787d145 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:13.147674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587297c-100e-404c-88a5-724c0787d145 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:13.231558 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1581a-6baa-49b0-8f8b-2ded48de99ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:13.232114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1581a-6baa-49b0-8f8b-2ded48de99ee admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:13.232438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1581a-6baa-49b0-8f8b-2ded48de99ee admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:13.232811 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1581a-6baa-49b0-8f8b-2ded48de99ee admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:13.233600 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1581a-6baa-49b0-8f8b-2ded48de99ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:13.235973 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1581a-6baa-49b0-8f8b-2ded48de99ee admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:13.358277 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1252/2513] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:41:13 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:41:13.363421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ffcb7-6693-4bc5-be9e-b52917e99939 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:13.381145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ffcb7-6693-4bc5-be9e-b52917e99939 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:13.387589 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98f0ab-c52a-4d22-b21c-64fe6ebb7b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:13.388360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98f0ab-c52a-4d22-b21c-64fe6ebb7b6c admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:13.388558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98f0ab-c52a-4d22-b21c-64fe6ebb7b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:13.388811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98f0ab-c52a-4d22-b21c-64fe6ebb7b6c admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:13.389485 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98f0ab-c52a-4d22-b21c-64fe6ebb7b6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:13.391321 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98f0ab-c52a-4d22-b21c-64fe6ebb7b6c admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:13.406610 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1262/2514] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:41:13.411649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1581a-6baa-49b0-8f8b-2ded48de99ee admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:13.427969 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1581a-6baa-49b0-8f8b-2ded48de99ee admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:13.434157 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3fc06-ac7c-400f-a137-87099bb64c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:13.434957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3fc06-ac7c-400f-a137-87099bb64c7b admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:13.435366 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3fc06-ac7c-400f-a137-87099bb64c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:13.435706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3fc06-ac7c-400f-a137-87099bb64c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:13.436442 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3fc06-ac7c-400f-a137-87099bb64c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:13.438184 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3fc06-ac7c-400f-a137-87099bb64c7b admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:13.457118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1253/2515] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:41:13.474388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98f0ab-c52a-4d22-b21c-64fe6ebb7b6c admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:13.501022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98f0ab-c52a-4d22-b21c-64fe6ebb7b6c admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:13.509450 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeaff9-2ebe-4a95-b166-d6216ef2a255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:13.510635 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeaff9-2ebe-4a95-b166-d6216ef2a255 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:13.511670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeaff9-2ebe-4a95-b166-d6216ef2a255 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:13.512169 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeaff9-2ebe-4a95-b166-d6216ef2a255 admin admin] PATH_INFO: `/v3/projects/652162682a1a4ab3b8dc548401f7dfe3/users/2256b87107284890a7c1ac15fd26a0aa/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:13.568233 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eeaff9-2ebe-4a95-b166-d6216ef2a255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=652162682a1a4ab3b8dc548401f7dfe3, user_id=2256b87107284890a7c1ac15fd26a0aa, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:13.574237 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eeaff9-2ebe-4a95-b166-d6216ef2a255 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:13.697368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1263/2516] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:41:13 2022] PUT /identity/v3/projects/652162682a1a4ab3b8dc548401f7dfe3/users/2256b87107284890a7c1ac15fd26a0aa/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:13.710899 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a0e52-aae7-4a10-886c-86e541c09059 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:13.711722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a0e52-aae7-4a10-886c-86e541c09059 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:13.711722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a0e52-aae7-4a10-886c-86e541c09059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:13.759207 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-072a0e52-aae7-4a10-886c-86e541c09059 None None] MFA Rules not processed for user `2256b87107284890a7c1ac15fd26a0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:13.759207 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-072a0e52-aae7-4a10-886c-86e541c09059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:13.770885 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeaff9-2ebe-4a95-b166-d6216ef2a255 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:13.774678 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-40eeaff9-2ebe-4a95-b166-d6216ef2a255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:13.844851 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:13.885226 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeaff9-2ebe-4a95-b166-d6216ef2a255 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:13.890546 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07a1c4-23e2-4ec1-933f-c23a09b9c4ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:13.891445 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07a1c4-23e2-4ec1-933f-c23a09b9c4ae service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:13.891704 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07a1c4-23e2-4ec1-933f-c23a09b9c4ae service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:13.892010 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07a1c4-23e2-4ec1-933f-c23a09b9c4ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:13.896201 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-5b07a1c4-23e2-4ec1-933f-c23a09b9c4ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:13.911418 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1254/2517] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:13.916345 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a0e52-aae7-4a10-886c-86e541c09059 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:14.049845 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07a1c4-23e2-4ec1-933f-c23a09b9c4ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:14.052906 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07a1c4-23e2-4ec1-933f-c23a09b9c4ae service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:14.072274 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a0e52-aae7-4a10-886c-86e541c09059 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:14.075452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1264/2518] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:41:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:41:14.082048 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d862f-fb1c-4218-a15b-c73ec909e1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:14.083341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d862f-fb1c-4218-a15b-c73ec909e1fc admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:14.083851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d862f-fb1c-4218-a15b-c73ec909e1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:14.084298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d862f-fb1c-4218-a15b-c73ec909e1fc admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:14.085487 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d862f-fb1c-4218-a15b-c73ec909e1fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:14.089810 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d862f-fb1c-4218-a15b-c73ec909e1fc admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:14.099934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:14.228538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1255/2519] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:41:13 2022] POST /identity/v3/projects => generated 371 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:41:14.238775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07a1c4-23e2-4ec1-933f-c23a09b9c4ae service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:14.254602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07a1c4-23e2-4ec1-933f-c23a09b9c4ae service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:14.375177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de4d46-7c93-4cb4-8c8d-e7e50d657f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:14.376256 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de4d46-7c93-4cb4-8c8d-e7e50d657f19 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:14.376804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de4d46-7c93-4cb4-8c8d-e7e50d657f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:14.377354 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de4d46-7c93-4cb4-8c8d-e7e50d657f19 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:14.378626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de4d46-7c93-4cb4-8c8d-e7e50d657f19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:14.382540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de4d46-7c93-4cb4-8c8d-e7e50d657f19 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:14.571502 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1265/2520] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:41:14 2022] POST /identity/v3/users => generated 292 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:41:14.578726 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d862f-fb1c-4218-a15b-c73ec909e1fc admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:14.595941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d862f-fb1c-4218-a15b-c73ec909e1fc admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:14.603054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59deab3-6536-4d56-a5d8-a34285137c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:14.604142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59deab3-6536-4d56-a5d8-a34285137c20 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:14.604671 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59deab3-6536-4d56-a5d8-a34285137c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:14.605123 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59deab3-6536-4d56-a5d8-a34285137c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:14.606206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59deab3-6536-4d56-a5d8-a34285137c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:14.607949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59deab3-6536-4d56-a5d8-a34285137c20 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:14.621473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1256/2521] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:41:14.628717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de4d46-7c93-4cb4-8c8d-e7e50d657f19 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:14.651206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de4d46-7c93-4cb4-8c8d-e7e50d657f19 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:14.661325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e436c5f-85c6-4a48-9fff-5b37f71adbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:14.662513 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e436c5f-85c6-4a48-9fff-5b37f71adbef admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:14.663088 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e436c5f-85c6-4a48-9fff-5b37f71adbef admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:14.663585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e436c5f-85c6-4a48-9fff-5b37f71adbef admin admin] PATH_INFO: `/v3/projects/54d2f0a81890433987e437afc8157cfb/users/445808ae2bfd4c94a4d8a1b71a59a5af/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:14.699214 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e436c5f-85c6-4a48-9fff-5b37f71adbef admin admin] RBAC: Authorizing `identity:create_grant(project_id=54d2f0a81890433987e437afc8157cfb, user_id=445808ae2bfd4c94a4d8a1b71a59a5af, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:14.704783 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e436c5f-85c6-4a48-9fff-5b37f71adbef admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:14.917775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1266/2522] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:41:14 2022] PUT /identity/v3/projects/54d2f0a81890433987e437afc8157cfb/users/445808ae2bfd4c94a4d8a1b71a59a5af/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:14.922792 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59deab3-6536-4d56-a5d8-a34285137c20 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:14.936023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59deab3-6536-4d56-a5d8-a34285137c20 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:15.047392 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9b632-467e-4df8-9fb5-9d3691957f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:15.048128 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9b632-467e-4df8-9fb5-9d3691957f6c admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:15.048424 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9b632-467e-4df8-9fb5-9d3691957f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:15.048702 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9b632-467e-4df8-9fb5-9d3691957f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:15.049528 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9b632-467e-4df8-9fb5-9d3691957f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:15.051447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9b632-467e-4df8-9fb5-9d3691957f6c admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:15.065968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1257/2523] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:41:15.076062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e436c5f-85c6-4a48-9fff-5b37f71adbef admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:15.101406 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e436c5f-85c6-4a48-9fff-5b37f71adbef admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:15.112399 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a082a-84f5-4ef2-a478-c73a42570dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:15.114306 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a082a-84f5-4ef2-a478-c73a42570dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:15.115466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a082a-84f5-4ef2-a478-c73a42570dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:15.116295 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a082a-84f5-4ef2-a478-c73a42570dc0 admin admin] PATH_INFO: `/v3/projects/54d2f0a81890433987e437afc8157cfb/users/445808ae2bfd4c94a4d8a1b71a59a5af/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:15.121539 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a082a-84f5-4ef2-a478-c73a42570dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54d2f0a81890433987e437afc8157cfb, user_id=445808ae2bfd4c94a4d8a1b71a59a5af, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:15.128211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a082a-84f5-4ef2-a478-c73a42570dc0 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:15.155553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1267/2524] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:41:15 2022] PUT /identity/v3/projects/54d2f0a81890433987e437afc8157cfb/users/445808ae2bfd4c94a4d8a1b71a59a5af/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:15.185075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2e0b7-3b2e-408f-bda4-9c1b48363f41 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:15.185666 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2e0b7-3b2e-408f-bda4-9c1b48363f41 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:15.186017 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2e0b7-3b2e-408f-bda4-9c1b48363f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:15.260440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-d8e2e0b7-3b2e-408f-bda4-9c1b48363f41 None None] MFA Rules not processed for user `445808ae2bfd4c94a4d8a1b71a59a5af`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:15.269412 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d8e2e0b7-3b2e-408f-bda4-9c1b48363f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:15.394326 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1258/2525] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:15.413013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a082a-84f5-4ef2-a478-c73a42570dc0 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:15.415685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a87a082a-84f5-4ef2-a478-c73a42570dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:15.488750 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:15.523364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a082a-84f5-4ef2-a478-c73a42570dc0 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:15.535772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0a70a-52e2-477a-acb7-da9216d5e5d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:15.536935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0a70a-52e2-477a-acb7-da9216d5e5d9 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:15.537457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0a70a-52e2-477a-acb7-da9216d5e5d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:15.537952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0a70a-52e2-477a-acb7-da9216d5e5d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:15.554288 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0a70a-52e2-477a-acb7-da9216d5e5d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:15.555821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0a70a-52e2-477a-acb7-da9216d5e5d9 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:15.568397 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1268/2526] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:41:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:41:15.929211 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2e0b7-3b2e-408f-bda4-9c1b48363f41 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:15.951516 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2e0b7-3b2e-408f-bda4-9c1b48363f41 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:15.958955 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13572b3d-9f7b-487b-a529-a0786c2a29d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:15.960146 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13572b3d-9f7b-487b-a529-a0786c2a29d6 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:15.960705 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13572b3d-9f7b-487b-a529-a0786c2a29d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:15.962453 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13572b3d-9f7b-487b-a529-a0786c2a29d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:16.089090 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:16.091930 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13572b3d-9f7b-487b-a529-a0786c2a29d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:16.094871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13572b3d-9f7b-487b-a529-a0786c2a29d6 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:16.112373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1259/2527] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:41:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:41:18.989667 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3547d5a-5e7f-43fd-bb9f-aed5135a4e22 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:18.990530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3547d5a-5e7f-43fd-bb9f-aed5135a4e22 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:18.990697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3547d5a-5e7f-43fd-bb9f-aed5135a4e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:19.020581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-a3547d5a-5e7f-43fd-bb9f-aed5135a4e22 None None] MFA Rules not processed for user `9dae215cd92f46d18a3c1b7389c3af4b`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:19.026412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a3547d5a-5e7f-43fd-bb9f-aed5135a4e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:19.028399 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13572b3d-9f7b-487b-a529-a0786c2a29d6 service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:19.030933 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-13572b3d-9f7b-487b-a529-a0786c2a29d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:19.105205 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1269/2528] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:19.162606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13572b3d-9f7b-487b-a529-a0786c2a29d6 service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:19.173626 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a4cfd-7f72-4ad8-8f6a-94ccef831182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:19.173626 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a4cfd-7f72-4ad8-8f6a-94ccef831182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:19.173626 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a4cfd-7f72-4ad8-8f6a-94ccef831182 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:19.174333 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a4cfd-7f72-4ad8-8f6a-94ccef831182 admin admin] PATH_INFO: `/v3/users/ecc8d9701e5349f1a34e7ceae386384e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:19.176704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a4cfd-7f72-4ad8-8f6a-94ccef831182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecc8d9701e5349f1a34e7ceae386384e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:19.180760 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a4cfd-7f72-4ad8-8f6a-94ccef831182 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:19.506673 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d92a4cfd-7f72-4ad8-8f6a-94ccef831182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecc8d9701e5349f1a34e7ceae386384e'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:19.561031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d92a4cfd-7f72-4ad8-8f6a-94ccef831182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecc8d9701e5349f1a34e7ceae386384e'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:19.561994 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d92a4cfd-7f72-4ad8-8f6a-94ccef831182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecc8d9701e5349f1a34e7ceae386384e'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:19.579516 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d92a4cfd-7f72-4ad8-8f6a-94ccef831182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecc8d9701e5349f1a34e7ceae386384e'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:19.580700 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-d92a4cfd-7f72-4ad8-8f6a-94ccef831182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecc8d9701e5349f1a34e7ceae386384e'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:19.731640 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1260/2529] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:41:19 2022] DELETE /identity/v3/users/ecc8d9701e5349f1a34e7ceae386384e => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:19.734427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258bcb4-7eed-4685-890a-997edc4ed388 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:19.735293 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258bcb4-7eed-4685-890a-997edc4ed388 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:19.736008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258bcb4-7eed-4685-890a-997edc4ed388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:19.766852 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-0258bcb4-7eed-4685-890a-997edc4ed388 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:19.768455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0258bcb4-7eed-4685-890a-997edc4ed388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:19.875952 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1270/2530] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:41:19.912283 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a4cfd-7f72-4ad8-8f6a-94ccef831182 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:19.914540 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d92a4cfd-7f72-4ad8-8f6a-94ccef831182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:20.024847 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a4cfd-7f72-4ad8-8f6a-94ccef831182 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:20.031892 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21289a49-79a1-4dbc-a25a-ccf89959b1fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:20.032722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21289a49-79a1-4dbc-a25a-ccf89959b1fb service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:20.032869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21289a49-79a1-4dbc-a25a-ccf89959b1fb service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:20.033139 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21289a49-79a1-4dbc-a25a-ccf89959b1fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:20.069411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21289a49-79a1-4dbc-a25a-ccf89959b1fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:20.069411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21289a49-79a1-4dbc-a25a-ccf89959b1fb service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:20.096876 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1261/2531] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:41:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:41:20.903038 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258bcb4-7eed-4685-890a-997edc4ed388 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:20.905281 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0258bcb4-7eed-4685-890a-997edc4ed388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:20.951218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258bcb4-7eed-4685-890a-997edc4ed388 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:20.978450 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:20.981050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:20.982341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:20.984594 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f admin admin] PATH_INFO: `/v3/projects/d27d7514f1384589a91816bc0cbd8c1a` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:20.989618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d27d7514f1384589a91816bc0cbd8c1a)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:20.999519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:21.410464 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd27d7514f1384589a91816bc0cbd8c1a', 'request_id': 'req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:21.421601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd27d7514f1384589a91816bc0cbd8c1a', 'request_id': 'req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:21.556957 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd27d7514f1384589a91816bc0cbd8c1a', 'request_id': 'req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:21.558195 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21289a49-79a1-4dbc-a25a-ccf89959b1fb service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:21.561044 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-21289a49-79a1-4dbc-a25a-ccf89959b1fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:21.587155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21289a49-79a1-4dbc-a25a-ccf89959b1fb service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:21.597806 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f admin admin] The token cache is being invalidate because project d27d7514f1384589a91816bc0cbd8c1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:41:21.598084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:21.601052 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:21.611002 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1271/2532] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:41:20 2022] DELETE /identity/v3/projects/d27d7514f1384589a91816bc0cbd8c1a => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:41:21.616660 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:21.622878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:21.623499 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:21.642750 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:21.718912 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6a4c9-ec37-4edf-a1e0-4b7934b38c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:21.719632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6a4c9-ec37-4edf-a1e0-4b7934b38c0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:21.719905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6a4c9-ec37-4edf-a1e0-4b7934b38c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:21.720190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6a4c9-ec37-4edf-a1e0-4b7934b38c0b admin admin] PATH_INFO: `/v3/users/32401931d75649018f6b0e55144bb5f2` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:21.721542 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6a4c9-ec37-4edf-a1e0-4b7934b38c0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=32401931d75649018f6b0e55144bb5f2)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:21.723880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6a4c9-ec37-4edf-a1e0-4b7934b38c0b admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:21.753342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d205b4e-a967-4bc7-b8ac-46fbb08d052f admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:21.765511 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666ec3e-a00f-4fef-a977-8bdd0ec15b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:21.766479 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666ec3e-a00f-4fef-a977-8bdd0ec15b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:21.766967 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666ec3e-a00f-4fef-a977-8bdd0ec15b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:21.767505 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666ec3e-a00f-4fef-a977-8bdd0ec15b3d admin admin] PATH_INFO: `/v3/users/fbcd04b1fcc948a7a01c91692a7664be` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:21.769667 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666ec3e-a00f-4fef-a977-8bdd0ec15b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbcd04b1fcc948a7a01c91692a7664be)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:21.773626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666ec3e-a00f-4fef-a977-8bdd0ec15b3d admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:22.352962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f6a6a4c9-ec37-4edf-a1e0-4b7934b38c0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32401931d75649018f6b0e55144bb5f2'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:22.374284 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f6a6a4c9-ec37-4edf-a1e0-4b7934b38c0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32401931d75649018f6b0e55144bb5f2'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:22.377588 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f6a6a4c9-ec37-4edf-a1e0-4b7934b38c0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32401931d75649018f6b0e55144bb5f2'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:22.384724 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f6a6a4c9-ec37-4edf-a1e0-4b7934b38c0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32401931d75649018f6b0e55144bb5f2'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:22.386129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f6a6a4c9-ec37-4edf-a1e0-4b7934b38c0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32401931d75649018f6b0e55144bb5f2'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:22.425989 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e666ec3e-a00f-4fef-a977-8bdd0ec15b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbcd04b1fcc948a7a01c91692a7664be'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:22.492317 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1262/2533] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:41:21 2022] DELETE /identity/v3/users/32401931d75649018f6b0e55144bb5f2 => generated 0 bytes in 935 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:22.570202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e666ec3e-a00f-4fef-a977-8bdd0ec15b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbcd04b1fcc948a7a01c91692a7664be'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:22.571323 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e666ec3e-a00f-4fef-a977-8bdd0ec15b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbcd04b1fcc948a7a01c91692a7664be'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:22.572350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e666ec3e-a00f-4fef-a977-8bdd0ec15b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbcd04b1fcc948a7a01c91692a7664be'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:22.589334 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e666ec3e-a00f-4fef-a977-8bdd0ec15b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbcd04b1fcc948a7a01c91692a7664be'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:22.617783 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1272/2534] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:41:21 2022] DELETE /identity/v3/users/fbcd04b1fcc948a7a01c91692a7664be => generated 0 bytes in 1002 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:22.874414 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6a4c9-ec37-4edf-a1e0-4b7934b38c0b admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:22.876585 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f6a6a4c9-ec37-4edf-a1e0-4b7934b38c0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:22.986520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6a4c9-ec37-4edf-a1e0-4b7934b38c0b admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:23.097542 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6eaf7-6a96-4980-8736-12b0513d1b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:23.098472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6eaf7-6a96-4980-8736-12b0513d1b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:23.098759 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6eaf7-6a96-4980-8736-12b0513d1b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:23.099190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6eaf7-6a96-4980-8736-12b0513d1b66 admin admin] PATH_INFO: `/v3/projects/6a627a9e151e4e8d8522afbc844c550c` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:23.100591 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6eaf7-6a96-4980-8736-12b0513d1b66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a627a9e151e4e8d8522afbc844c550c)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:23.103243 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6eaf7-6a96-4980-8736-12b0513d1b66 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:23.234685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-bfd6eaf7-6a96-4980-8736-12b0513d1b66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a627a9e151e4e8d8522afbc844c550c', 'request_id': 'req-bfd6eaf7-6a96-4980-8736-12b0513d1b66'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:23.242987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-bfd6eaf7-6a96-4980-8736-12b0513d1b66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a627a9e151e4e8d8522afbc844c550c', 'request_id': 'req-bfd6eaf7-6a96-4980-8736-12b0513d1b66'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:23.253378 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666ec3e-a00f-4fef-a977-8bdd0ec15b3d admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:23.256009 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e666ec3e-a00f-4fef-a977-8bdd0ec15b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:23.277511 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666ec3e-a00f-4fef-a977-8bdd0ec15b3d admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:23.350363 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-bfd6eaf7-6a96-4980-8736-12b0513d1b66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a627a9e151e4e8d8522afbc844c550c', 'request_id': 'req-bfd6eaf7-6a96-4980-8736-12b0513d1b66'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:23.380867 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-bfd6eaf7-6a96-4980-8736-12b0513d1b66 admin admin] The token cache is being invalidate because project 6a627a9e151e4e8d8522afbc844c550c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:41:23.381295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-bfd6eaf7-6a96-4980-8736-12b0513d1b66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:23.383323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-bfd6eaf7-6a96-4980-8736-12b0513d1b66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:23.391142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1263/2535] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:41:22 2022] DELETE /identity/v3/projects/6a627a9e151e4e8d8522afbc844c550c => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:41:23.394238 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc496ac-622d-454f-85ba-141e6e80cf57 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:23.394697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc496ac-622d-454f-85ba-141e6e80cf57 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:23.395164 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc496ac-622d-454f-85ba-141e6e80cf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:23.395446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e18131-b7ef-40c5-b519-f00158c542b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:23.396524 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e18131-b7ef-40c5-b519-f00158c542b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:23.397100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e18131-b7ef-40c5-b519-f00158c542b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:23.397521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e18131-b7ef-40c5-b519-f00158c542b6 admin admin] PATH_INFO: `/v3/projects/8147f57388d04bf4b1e82c3817b9db10` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:23.400239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e18131-b7ef-40c5-b519-f00158c542b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8147f57388d04bf4b1e82c3817b9db10)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:23.406799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e18131-b7ef-40c5-b519-f00158c542b6 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:23.427421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-4fc496ac-622d-454f-85ba-141e6e80cf57 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:23.431359 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-4fc496ac-622d-454f-85ba-141e6e80cf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:23.448985 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1264/2536] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:41:23.459349 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f741b-a2d8-4a31-afd0-b3f355c2f767 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:23.459844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f741b-a2d8-4a31-afd0-b3f355c2f767 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:23.460524 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f741b-a2d8-4a31-afd0-b3f355c2f767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:23.488413 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c81f741b-a2d8-4a31-afd0-b3f355c2f767 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:23.489693 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c81f741b-a2d8-4a31-afd0-b3f355c2f767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:23.500082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1265/2537] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:23.505145 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f741b-a2d8-4a31-afd0-b3f355c2f767 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:23.517889 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f741b-a2d8-4a31-afd0-b3f355c2f767 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:23.523170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f2fd7-a2bf-4804-b82e-0e9484a3b274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:23.524031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f2fd7-a2bf-4804-b82e-0e9484a3b274 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:23.524312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f2fd7-a2bf-4804-b82e-0e9484a3b274 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:23.524585 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f2fd7-a2bf-4804-b82e-0e9484a3b274 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:23.525402 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f2fd7-a2bf-4804-b82e-0e9484a3b274 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:41:23.525762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f2fd7-a2bf-4804-b82e-0e9484a3b274 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:23.528275 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f2fd7-a2bf-4804-b82e-0e9484a3b274 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:23.546795 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1266/2538] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:41:23.555042 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f2fd7-a2bf-4804-b82e-0e9484a3b274 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:23.568664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-f8e18131-b7ef-40c5-b519-f00158c542b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8147f57388d04bf4b1e82c3817b9db10', 'request_id': 'req-f8e18131-b7ef-40c5-b519-f00158c542b6'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:23.571829 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f2fd7-a2bf-4804-b82e-0e9484a3b274 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:23.578909 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-f8e18131-b7ef-40c5-b519-f00158c542b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8147f57388d04bf4b1e82c3817b9db10', 'request_id': 'req-f8e18131-b7ef-40c5-b519-f00158c542b6'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:23.648505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca72940-f675-43a0-8767-3a5a7cfd96db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:23.649290 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca72940-f675-43a0-8767-3a5a7cfd96db admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:23.649621 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca72940-f675-43a0-8767-3a5a7cfd96db admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:23.649954 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca72940-f675-43a0-8767-3a5a7cfd96db admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:23.650674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca72940-f675-43a0-8767-3a5a7cfd96db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:23.654945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca72940-f675-43a0-8767-3a5a7cfd96db admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:23.721978 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-f8e18131-b7ef-40c5-b519-f00158c542b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8147f57388d04bf4b1e82c3817b9db10', 'request_id': 'req-f8e18131-b7ef-40c5-b519-f00158c542b6'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:23.742035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-f8e18131-b7ef-40c5-b519-f00158c542b6 admin admin] The token cache is being invalidate because project 8147f57388d04bf4b1e82c3817b9db10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:41:23.742493 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-f8e18131-b7ef-40c5-b519-f00158c542b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:23.743634 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-f8e18131-b7ef-40c5-b519-f00158c542b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:23.746302 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1273/2539] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:41:23 2022] DELETE /identity/v3/projects/8147f57388d04bf4b1e82c3817b9db10 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:41:23.754583 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbeece-035c-421b-ac34-24a84c4c8631 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:23.755100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbeece-035c-421b-ac34-24a84c4c8631 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:23.755597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbeece-035c-421b-ac34-24a84c4c8631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:23.793598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-2fdbeece-035c-421b-ac34-24a84c4c8631 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:23.796021 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-2fdbeece-035c-421b-ac34-24a84c4c8631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:23.811626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1274/2540] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:23.827414 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47333d6-3c64-4cc3-a633-2a511dec980c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:23.827862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47333d6-3c64-4cc3-a633-2a511dec980c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:23.828280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47333d6-3c64-4cc3-a633-2a511dec980c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:23.832457 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1267/2541] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:41:23 2022] POST /identity/v3/projects => generated 385 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:41:23.833915 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca72940-f675-43a0-8767-3a5a7cfd96db admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:23.836145 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-bca72940-f675-43a0-8767-3a5a7cfd96db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:23.863512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-e47333d6-3c64-4cc3-a633-2a511dec980c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:23.865057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e47333d6-3c64-4cc3-a633-2a511dec980c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:23.901735 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:23.938683 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca72940-f675-43a0-8767-3a5a7cfd96db admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:23.944404 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fca8a8-f503-4c0e-9cc5-b67ee6805196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:23.945086 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fca8a8-f503-4c0e-9cc5-b67ee6805196 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:23.945472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fca8a8-f503-4c0e-9cc5-b67ee6805196 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:23.945761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fca8a8-f503-4c0e-9cc5-b67ee6805196 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:23.946509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fca8a8-f503-4c0e-9cc5-b67ee6805196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:23.948782 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fca8a8-f503-4c0e-9cc5-b67ee6805196 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:23.963032 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1275/2542] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:23.969552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47333d6-3c64-4cc3-a633-2a511dec980c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:23.982352 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47333d6-3c64-4cc3-a633-2a511dec980c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:23.987757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd57a8-3843-4d93-9b49-218d25f376cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:23.988452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd57a8-3843-4d93-9b49-218d25f376cf admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:23.988734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd57a8-3843-4d93-9b49-218d25f376cf admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:23.989011 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd57a8-3843-4d93-9b49-218d25f376cf admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:23.989711 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd57a8-3843-4d93-9b49-218d25f376cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:41:23.990016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd57a8-3843-4d93-9b49-218d25f376cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:23.991820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd57a8-3843-4d93-9b49-218d25f376cf admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:24.008865 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1276/2543] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:41:24.016762 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd57a8-3843-4d93-9b49-218d25f376cf admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:24.029078 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd57a8-3843-4d93-9b49-218d25f376cf admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:24.034803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a919470-c096-4ba4-98ba-da07bb165307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:24.035857 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a919470-c096-4ba4-98ba-da07bb165307 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:24.036511 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a919470-c096-4ba4-98ba-da07bb165307 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:24.038474 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:24.039352 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a919470-c096-4ba4-98ba-da07bb165307 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:24.042040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a919470-c096-4ba4-98ba-da07bb165307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:24.042712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a919470-c096-4ba4-98ba-da07bb165307 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:24.078456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1268/2544] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:41:23 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:41:24.080719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fca8a8-f503-4c0e-9cc5-b67ee6805196 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:24.093187 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fca8a8-f503-4c0e-9cc5-b67ee6805196 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:24.098967 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee029a-702a-4c71-944a-f5b4710f0630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:24.099755 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee029a-702a-4c71-944a-f5b4710f0630 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:24.100113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee029a-702a-4c71-944a-f5b4710f0630 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:24.100456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee029a-702a-4c71-944a-f5b4710f0630 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:24.101307 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee029a-702a-4c71-944a-f5b4710f0630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:24.103265 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee029a-702a-4c71-944a-f5b4710f0630 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:24.116218 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1269/2545] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:41:24.122015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee029a-702a-4c71-944a-f5b4710f0630 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:24.133991 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee029a-702a-4c71-944a-f5b4710f0630 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:24.140786 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c35c8-0034-43f0-b7ba-b7f30ee82d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:24.141485 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c35c8-0034-43f0-b7ba-b7f30ee82d80 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:24.141769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c35c8-0034-43f0-b7ba-b7f30ee82d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:24.142207 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c35c8-0034-43f0-b7ba-b7f30ee82d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:24.142888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c35c8-0034-43f0-b7ba-b7f30ee82d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:24.144676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c35c8-0034-43f0-b7ba-b7f30ee82d80 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:24.157943 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1277/2546] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:41:24 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:41:24.164499 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1270/2547] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:41:24.166211 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c35c8-0034-43f0-b7ba-b7f30ee82d80 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:24.166429 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a919470-c096-4ba4-98ba-da07bb165307 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:24.178223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c35c8-0034-43f0-b7ba-b7f30ee82d80 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:24.247686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a919470-c096-4ba4-98ba-da07bb165307 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:24.255123 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27fb97-d975-467d-8338-1f1bf3993513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:24.255937 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27fb97-d975-467d-8338-1f1bf3993513 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:24.256353 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27fb97-d975-467d-8338-1f1bf3993513 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:24.256836 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27fb97-d975-467d-8338-1f1bf3993513 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:24.257038 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b71f7-79fa-401a-9be7-a7c3627d0b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:24.257591 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b71f7-79fa-401a-9be7-a7c3627d0b2f admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:24.257862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27fb97-d975-467d-8338-1f1bf3993513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:24.258035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b71f7-79fa-401a-9be7-a7c3627d0b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:24.258206 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b71f7-79fa-401a-9be7-a7c3627d0b2f admin admin] PATH_INFO: `/v3/projects/09755c2e669e40aeb5115662efa0a2e5/users/1b1864fe53964be88612f99c8b26b772/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:24.260514 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27fb97-d975-467d-8338-1f1bf3993513 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:24.279219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b71f7-79fa-401a-9be7-a7c3627d0b2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=09755c2e669e40aeb5115662efa0a2e5, user_id=1b1864fe53964be88612f99c8b26b772, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:24.282419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b71f7-79fa-401a-9be7-a7c3627d0b2f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:24.394496 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1271/2548] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:41:24 2022] PUT /identity/v3/projects/09755c2e669e40aeb5115662efa0a2e5/users/1b1864fe53964be88612f99c8b26b772/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:41:24.396206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1278/2549] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:41:24 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:41:24.400854 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b71f7-79fa-401a-9be7-a7c3627d0b2f admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:24.407431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ff99c-b473-441d-8775-ebe810328cf2 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:24.407841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ff99c-b473-441d-8775-ebe810328cf2 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:24.408179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ff99c-b473-441d-8775-ebe810328cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:24.446685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-0f6ff99c-b473-441d-8775-ebe810328cf2 None None] MFA Rules not processed for user `1b1864fe53964be88612f99c8b26b772`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:24.448732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0f6ff99c-b473-441d-8775-ebe810328cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:24.472555 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:24.490532 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b71f7-79fa-401a-9be7-a7c3627d0b2f admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:24.507997 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:24.513642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980e61f-b614-49ea-af6a-3fd098258eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:24.513642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980e61f-b614-49ea-af6a-3fd098258eea admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:24.514488 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980e61f-b614-49ea-af6a-3fd098258eea admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:24.514595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980e61f-b614-49ea-af6a-3fd098258eea admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:24.515908 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980e61f-b614-49ea-af6a-3fd098258eea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:24.519219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980e61f-b614-49ea-af6a-3fd098258eea admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:24.550200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1272/2550] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:41:24.551484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980e61f-b614-49ea-af6a-3fd098258eea admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:24.567769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980e61f-b614-49ea-af6a-3fd098258eea admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:24.578406 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b84bd-a4e8-43df-ab3d-eaf91ff636f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:24.580030 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b84bd-a4e8-43df-ab3d-eaf91ff636f4 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:24.580030 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b84bd-a4e8-43df-ab3d-eaf91ff636f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:24.580030 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b84bd-a4e8-43df-ab3d-eaf91ff636f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:24.581055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b84bd-a4e8-43df-ab3d-eaf91ff636f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:24.583549 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b84bd-a4e8-43df-ab3d-eaf91ff636f4 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:24.597897 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1279/2551] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:41:24.610351 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ff99c-b473-441d-8775-ebe810328cf2 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:24.611988 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0f6ff99c-b473-441d-8775-ebe810328cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:24.613520 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1273/2552] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:41:24.622516 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b84bd-a4e8-43df-ab3d-eaf91ff636f4 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:24.633383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b84bd-a4e8-43df-ab3d-eaf91ff636f4 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:24.639126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e33f3e-72f4-4131-8141-4ade68906ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:24.639827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e33f3e-72f4-4131-8141-4ade68906ae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:24.640200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e33f3e-72f4-4131-8141-4ade68906ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:24.640472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e33f3e-72f4-4131-8141-4ade68906ae6 admin admin] PATH_INFO: `/v3/projects/b22f84a7fdc24440b22a2e0ba9fb11e1/users/4bd8a483dc6147258af1681ec0e8e4a3/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:24.660890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e33f3e-72f4-4131-8141-4ade68906ae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b22f84a7fdc24440b22a2e0ba9fb11e1, user_id=4bd8a483dc6147258af1681ec0e8e4a3, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:24.664176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e33f3e-72f4-4131-8141-4ade68906ae6 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:24.706040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ff99c-b473-441d-8775-ebe810328cf2 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:24.712661 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3133eb06-e53a-44a0-a7be-9ea576a7fa17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:24.713356 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133eb06-e53a-44a0-a7be-9ea576a7fa17 service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:24.713773 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133eb06-e53a-44a0-a7be-9ea576a7fa17 service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:24.714241 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3133eb06-e53a-44a0-a7be-9ea576a7fa17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:24.724240 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3133eb06-e53a-44a0-a7be-9ea576a7fa17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:24.727004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3133eb06-e53a-44a0-a7be-9ea576a7fa17 service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:24.743055 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1280/2553] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:41:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:41:24.821053 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1274/2554] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:41:24 2022] PUT /identity/v3/projects/b22f84a7fdc24440b22a2e0ba9fb11e1/users/4bd8a483dc6147258af1681ec0e8e4a3/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:24.843063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b07cf4-8e02-46ff-b77a-8d179bb87d03 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:24.843517 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b07cf4-8e02-46ff-b77a-8d179bb87d03 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:24.843953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b07cf4-8e02-46ff-b77a-8d179bb87d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:24.896150 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-e3b07cf4-8e02-46ff-b77a-8d179bb87d03 None None] MFA Rules not processed for user `4bd8a483dc6147258af1681ec0e8e4a3`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:24.900670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e3b07cf4-8e02-46ff-b77a-8d179bb87d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:24.990639 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:24.991694 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1281/2555] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:24.997831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e33f3e-72f4-4131-8141-4ade68906ae6 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:25.110179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e33f3e-72f4-4131-8141-4ade68906ae6 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:25.118616 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2709bd-b5ca-416a-917d-d95bfd2f2486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:25.120396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2709bd-b5ca-416a-917d-d95bfd2f2486 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:25.125884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2709bd-b5ca-416a-917d-d95bfd2f2486 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:25.125884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2709bd-b5ca-416a-917d-d95bfd2f2486 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:25.125884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2709bd-b5ca-416a-917d-d95bfd2f2486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:25.126351 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2709bd-b5ca-416a-917d-d95bfd2f2486 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:25.296642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1275/2556] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:41:24 2022] POST /identity/v3/projects => generated 377 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:41:25.301431 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b07cf4-8e02-46ff-b77a-8d179bb87d03 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:25.309619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:25.316753 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b07cf4-8e02-46ff-b77a-8d179bb87d03 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:25.428915 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2dbbca-dfa0-43cb-9c47-6ba747e2d022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:25.429881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dbbca-dfa0-43cb-9c47-6ba747e2d022 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:25.430434 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dbbca-dfa0-43cb-9c47-6ba747e2d022 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:25.430866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dbbca-dfa0-43cb-9c47-6ba747e2d022 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:25.431849 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2dbbca-dfa0-43cb-9c47-6ba747e2d022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:25.436682 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2dbbca-dfa0-43cb-9c47-6ba747e2d022 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:25.593360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1282/2557] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:41:25 2022] POST /identity/v3/users => generated 295 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:41:25.598851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2709bd-b5ca-416a-917d-d95bfd2f2486 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:25.616598 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2709bd-b5ca-416a-917d-d95bfd2f2486 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:25.624591 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc26d1c-3a9c-4b96-8d09-3658601ebf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:25.625633 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc26d1c-3a9c-4b96-8d09-3658601ebf34 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:25.626075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc26d1c-3a9c-4b96-8d09-3658601ebf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:25.626510 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc26d1c-3a9c-4b96-8d09-3658601ebf34 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:25.627769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc26d1c-3a9c-4b96-8d09-3658601ebf34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:25.630596 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc26d1c-3a9c-4b96-8d09-3658601ebf34 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:25.650328 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1276/2558] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:41:25.656969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2dbbca-dfa0-43cb-9c47-6ba747e2d022 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:25.671081 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2dbbca-dfa0-43cb-9c47-6ba747e2d022 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:25.679365 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28014234-46ae-4a67-8692-114fd6702fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:25.680879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28014234-46ae-4a67-8692-114fd6702fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:25.681346 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28014234-46ae-4a67-8692-114fd6702fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:25.681825 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28014234-46ae-4a67-8692-114fd6702fc8 admin admin] PATH_INFO: `/v3/projects/d5192e41fd7440db952fae68a130e66e/users/9cdbabe1186744c3bd3139bb767d1f30/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:25.709920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28014234-46ae-4a67-8692-114fd6702fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5192e41fd7440db952fae68a130e66e, user_id=9cdbabe1186744c3bd3139bb767d1f30, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:25.715841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28014234-46ae-4a67-8692-114fd6702fc8 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:25.825456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1283/2559] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:41:25 2022] PUT /identity/v3/projects/d5192e41fd7440db952fae68a130e66e/users/9cdbabe1186744c3bd3139bb767d1f30/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:41:25.829718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc26d1c-3a9c-4b96-8d09-3658601ebf34 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:25.852014 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc26d1c-3a9c-4b96-8d09-3658601ebf34 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:25.953104 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36540f-db6e-4641-ba5e-963a6f99b3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:25.953748 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36540f-db6e-4641-ba5e-963a6f99b3ac admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:25.954031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36540f-db6e-4641-ba5e-963a6f99b3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:25.954306 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36540f-db6e-4641-ba5e-963a6f99b3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:25.954991 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36540f-db6e-4641-ba5e-963a6f99b3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:25.956878 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36540f-db6e-4641-ba5e-963a6f99b3ac admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:25.969426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1277/2560] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:41:25.974032 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28014234-46ae-4a67-8692-114fd6702fc8 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:25.986498 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28014234-46ae-4a67-8692-114fd6702fc8 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:25.991628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c4f4e-ca79-42cc-aeaa-507a91bc06b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:25.992317 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c4f4e-ca79-42cc-aeaa-507a91bc06b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:25.992591 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c4f4e-ca79-42cc-aeaa-507a91bc06b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:25.992892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c4f4e-ca79-42cc-aeaa-507a91bc06b6 admin admin] PATH_INFO: `/v3/projects/d5192e41fd7440db952fae68a130e66e/users/9cdbabe1186744c3bd3139bb767d1f30/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:25.996443 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c4f4e-ca79-42cc-aeaa-507a91bc06b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5192e41fd7440db952fae68a130e66e, user_id=9cdbabe1186744c3bd3139bb767d1f30, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:25.999188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c4f4e-ca79-42cc-aeaa-507a91bc06b6 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:26.018524 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1284/2561] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:41:25 2022] PUT /identity/v3/projects/d5192e41fd7440db952fae68a130e66e/users/9cdbabe1186744c3bd3139bb767d1f30/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:26.025684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c568c03-40be-4fb9-8964-22319c4233f9 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:26.026319 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c568c03-40be-4fb9-8964-22319c4233f9 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:26.027007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c568c03-40be-4fb9-8964-22319c4233f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:26.062674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-6c568c03-40be-4fb9-8964-22319c4233f9 None None] MFA Rules not processed for user `9cdbabe1186744c3bd3139bb767d1f30`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:26.063754 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-6c568c03-40be-4fb9-8964-22319c4233f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:26.123618 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:26.173221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1278/2562] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:26.187408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c4f4e-ca79-42cc-aeaa-507a91bc06b6 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:26.189286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e06c4f4e-ca79-42cc-aeaa-507a91bc06b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:26.289674 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c4f4e-ca79-42cc-aeaa-507a91bc06b6 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:26.294476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e06e6d-08d4-4073-8ed6-14cc35cd544a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:26.295345 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e06e6d-08d4-4073-8ed6-14cc35cd544a service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:26.295730 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e06e6d-08d4-4073-8ed6-14cc35cd544a service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:26.296107 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e06e6d-08d4-4073-8ed6-14cc35cd544a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:26.378313 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e06e6d-08d4-4073-8ed6-14cc35cd544a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:26.379717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e06e6d-08d4-4073-8ed6-14cc35cd544a service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:26.393034 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1285/2563] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:41:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:41:36.332654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=10 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:36.387399 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=90,keystone:INSERT=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:37.708642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c568c03-40be-4fb9-8964-22319c4233f9 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:37.735805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c568c03-40be-4fb9-8964-22319c4233f9 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:37.743188 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e459e-97ef-4663-91af-09b8ace954da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:37.744736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e459e-97ef-4663-91af-09b8ace954da service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:37.745199 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e459e-97ef-4663-91af-09b8ace954da service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:37.746547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e459e-97ef-4663-91af-09b8ace954da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:37.765339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e459e-97ef-4663-91af-09b8ace954da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:37.767811 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e459e-97ef-4663-91af-09b8ace954da service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:37.785603 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1279/2564] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:41:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:41:39.587365 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e06e6d-08d4-4073-8ed6-14cc35cd544a service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:39.706414 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e06e6d-08d4-4073-8ed6-14cc35cd544a service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:39.714506 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b910a6b-c3ad-46bb-8b0f-3417d2195bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:39.715807 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b910a6b-c3ad-46bb-8b0f-3417d2195bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:39.716066 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b910a6b-c3ad-46bb-8b0f-3417d2195bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:39.716572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b910a6b-c3ad-46bb-8b0f-3417d2195bd8 admin admin] PATH_INFO: `/v3/users/1b1864fe53964be88612f99c8b26b772` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:39.718762 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b910a6b-c3ad-46bb-8b0f-3417d2195bd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b1864fe53964be88612f99c8b26b772)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:39.721881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b910a6b-c3ad-46bb-8b0f-3417d2195bd8 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:40.009686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-8b910a6b-c3ad-46bb-8b0f-3417d2195bd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b1864fe53964be88612f99c8b26b772'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:40.122696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-8b910a6b-c3ad-46bb-8b0f-3417d2195bd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b1864fe53964be88612f99c8b26b772'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:40.123782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-8b910a6b-c3ad-46bb-8b0f-3417d2195bd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b1864fe53964be88612f99c8b26b772'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:40.124830 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-8b910a6b-c3ad-46bb-8b0f-3417d2195bd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b1864fe53964be88612f99c8b26b772'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:40.140602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-8b910a6b-c3ad-46bb-8b0f-3417d2195bd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b1864fe53964be88612f99c8b26b772'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:40.173505 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1286/2565] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:41:39 2022] DELETE /identity/v3/users/1b1864fe53964be88612f99c8b26b772 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:40.179870 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e0a02-d644-497b-b001-589132627c61 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:40.180481 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e0a02-d644-497b-b001-589132627c61 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:40.181056 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e0a02-d644-497b-b001-589132627c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:40.215650 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-fc7e0a02-d644-497b-b001-589132627c61 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:40.217246 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-fc7e0a02-d644-497b-b001-589132627c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:40.305701 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1280/2566] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:40.321112 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b910a6b-c3ad-46bb-8b0f-3417d2195bd8 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:40.324773 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8b910a6b-c3ad-46bb-8b0f-3417d2195bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:40.435536 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b910a6b-c3ad-46bb-8b0f-3417d2195bd8 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:40.441629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62faee4a-ce76-4dd1-b73d-117392243965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:40.442638 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62faee4a-ce76-4dd1-b73d-117392243965 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:40.443172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62faee4a-ce76-4dd1-b73d-117392243965 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:40.443648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62faee4a-ce76-4dd1-b73d-117392243965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:40.460088 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62faee4a-ce76-4dd1-b73d-117392243965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:40.461841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62faee4a-ce76-4dd1-b73d-117392243965 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:40.483319 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1287/2567] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:41:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:41:41.142372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e0a02-d644-497b-b001-589132627c61 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:41.145124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-fc7e0a02-d644-497b-b001-589132627c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:41.180159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e0a02-d644-497b-b001-589132627c61 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:41.188290 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d984adc-d2fd-40a0-bf03-74710e065901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:41.189685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d984adc-d2fd-40a0-bf03-74710e065901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:41.190718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d984adc-d2fd-40a0-bf03-74710e065901 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:41.191569 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d984adc-d2fd-40a0-bf03-74710e065901 admin admin] PATH_INFO: `/v3/projects/09755c2e669e40aeb5115662efa0a2e5` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:41.193599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d984adc-d2fd-40a0-bf03-74710e065901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09755c2e669e40aeb5115662efa0a2e5)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:41.204934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d984adc-d2fd-40a0-bf03-74710e065901 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:41.395106 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2d984adc-d2fd-40a0-bf03-74710e065901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09755c2e669e40aeb5115662efa0a2e5', 'request_id': 'req-2d984adc-d2fd-40a0-bf03-74710e065901'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:41.409174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2d984adc-d2fd-40a0-bf03-74710e065901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09755c2e669e40aeb5115662efa0a2e5', 'request_id': 'req-2d984adc-d2fd-40a0-bf03-74710e065901'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:41.509387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2d984adc-d2fd-40a0-bf03-74710e065901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09755c2e669e40aeb5115662efa0a2e5', 'request_id': 'req-2d984adc-d2fd-40a0-bf03-74710e065901'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:41.526124 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2d984adc-d2fd-40a0-bf03-74710e065901 admin admin] The token cache is being invalidate because project 09755c2e669e40aeb5115662efa0a2e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:41:41.526784 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2d984adc-d2fd-40a0-bf03-74710e065901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:41.527698 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2d984adc-d2fd-40a0-bf03-74710e065901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:41:41.529323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1281/2568] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:41:41 2022] DELETE /identity/v3/projects/09755c2e669e40aeb5115662efa0a2e5 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:41.538038 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d63f7d-e2bf-4adf-9c8c-378a966b031f None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:41.538749 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d63f7d-e2bf-4adf-9c8c-378a966b031f None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:41.539060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d63f7d-e2bf-4adf-9c8c-378a966b031f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:41.574470 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-e5d63f7d-e2bf-4adf-9c8c-378a966b031f None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:41.576777 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e5d63f7d-e2bf-4adf-9c8c-378a966b031f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:41.609473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:41.694882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1288/2569] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:41.703743 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53f417-e100-47a6-a11f-545dba37a3ac None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:41.704107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53f417-e100-47a6-a11f-545dba37a3ac None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:41.704436 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53f417-e100-47a6-a11f-545dba37a3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:41.735863 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-0d53f417-e100-47a6-a11f-545dba37a3ac None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:41.737182 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-0d53f417-e100-47a6-a11f-545dba37a3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:41.747542 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1282/2570] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:41:41.752640 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d63f7d-e2bf-4adf-9c8c-378a966b031f None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:41.770517 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d63f7d-e2bf-4adf-9c8c-378a966b031f None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:41.779733 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8b5ff-1b03-4e55-98b1-5115d52aa7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:41.780823 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8b5ff-1b03-4e55-98b1-5115d52aa7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:41.781333 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8b5ff-1b03-4e55-98b1-5115d52aa7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:41.781811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8b5ff-1b03-4e55-98b1-5115d52aa7a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:41.783299 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8b5ff-1b03-4e55-98b1-5115d52aa7a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:41:41.783843 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8b5ff-1b03-4e55-98b1-5115d52aa7a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:41.786955 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8b5ff-1b03-4e55-98b1-5115d52aa7a8 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:41.812991 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1289/2571] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:41:41.820462 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d53f417-e100-47a6-a11f-545dba37a3ac None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:41.832769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d53f417-e100-47a6-a11f-545dba37a3ac None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:41.839630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614a289-0bba-42cb-a314-fe7850b16542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:41.840513 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614a289-0bba-42cb-a314-fe7850b16542 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:41.840844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614a289-0bba-42cb-a314-fe7850b16542 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:41.841271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614a289-0bba-42cb-a314-fe7850b16542 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:41.842043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614a289-0bba-42cb-a314-fe7850b16542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:41.844627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614a289-0bba-42cb-a314-fe7850b16542 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:42.022240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1283/2572] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:41:41 2022] POST /identity/v3/projects => generated 379 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:41:42.030099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8b5ff-1b03-4e55-98b1-5115d52aa7a8 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:42.050538 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8b5ff-1b03-4e55-98b1-5115d52aa7a8 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:42.165187 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3e021-1136-44ce-814b-27440d057e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:42.166281 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3e021-1136-44ce-814b-27440d057e4b admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:42.166764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3e021-1136-44ce-814b-27440d057e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:42.167290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3e021-1136-44ce-814b-27440d057e4b admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:42.168272 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3e021-1136-44ce-814b-27440d057e4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:42.171527 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3e021-1136-44ce-814b-27440d057e4b admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:42.248702 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1290/2573] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:41:42 2022] POST /identity/v3/users => generated 290 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:41:42.254379 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614a289-0bba-42cb-a314-fe7850b16542 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:42.273067 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614a289-0bba-42cb-a314-fe7850b16542 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:42.279285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01196f00-d33b-49ed-ba9c-6a88b314fabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:42.280143 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01196f00-d33b-49ed-ba9c-6a88b314fabc admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:42.280535 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01196f00-d33b-49ed-ba9c-6a88b314fabc admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:42.280924 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01196f00-d33b-49ed-ba9c-6a88b314fabc admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:42.281721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01196f00-d33b-49ed-ba9c-6a88b314fabc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:42.283888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01196f00-d33b-49ed-ba9c-6a88b314fabc admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:42.298860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1284/2574] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:41:42.304137 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3e021-1136-44ce-814b-27440d057e4b admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:42.318076 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3e021-1136-44ce-814b-27440d057e4b admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:42.325562 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e014c-7322-430e-a95b-d94537542ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:42.326342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e014c-7322-430e-a95b-d94537542ffa admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:42.326794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e014c-7322-430e-a95b-d94537542ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:42.327206 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e014c-7322-430e-a95b-d94537542ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:42.328194 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e014c-7322-430e-a95b-d94537542ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:42.329977 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e014c-7322-430e-a95b-d94537542ffa admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:42.351469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1291/2575] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:41:42.360173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01196f00-d33b-49ed-ba9c-6a88b314fabc admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:42.380095 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01196f00-d33b-49ed-ba9c-6a88b314fabc admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:42.388776 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24b0da-a8bb-4872-a041-ba7b8e205eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:42.389768 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24b0da-a8bb-4872-a041-ba7b8e205eca admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:42.390255 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24b0da-a8bb-4872-a041-ba7b8e205eca admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:42.390871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24b0da-a8bb-4872-a041-ba7b8e205eca admin admin] PATH_INFO: `/v3/projects/f10e26aa6b684a398ec7452e0356548e/users/0f1d9342be86475bad8132125a7836d7/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:42.422437 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24b0da-a8bb-4872-a041-ba7b8e205eca admin admin] RBAC: Authorizing `identity:create_grant(project_id=f10e26aa6b684a398ec7452e0356548e, user_id=0f1d9342be86475bad8132125a7836d7, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:42.428481 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24b0da-a8bb-4872-a041-ba7b8e205eca admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:42.515466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1285/2576] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:41:42 2022] PUT /identity/v3/projects/f10e26aa6b684a398ec7452e0356548e/users/0f1d9342be86475bad8132125a7836d7/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:42.531434 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772e92a-916d-47b0-b614-2429eeaecac3 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:42.532838 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772e92a-916d-47b0-b614-2429eeaecac3 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:42.533257 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772e92a-916d-47b0-b614-2429eeaecac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:42.577895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-3772e92a-916d-47b0-b614-2429eeaecac3 None None] MFA Rules not processed for user `0f1d9342be86475bad8132125a7836d7`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:42.580062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-3772e92a-916d-47b0-b614-2429eeaecac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:42.727383 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1292/2577] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:42.735034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24b0da-a8bb-4872-a041-ba7b8e205eca admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:42.748791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:42.847275 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24b0da-a8bb-4872-a041-ba7b8e205eca admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:42.856179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac34c12-e92c-4a88-9f23-5b684e59de6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:42.857218 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac34c12-e92c-4a88-9f23-5b684e59de6e admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:42.857692 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac34c12-e92c-4a88-9f23-5b684e59de6e admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:42.858171 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac34c12-e92c-4a88-9f23-5b684e59de6e admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:42.859399 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac34c12-e92c-4a88-9f23-5b684e59de6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:42.863235 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac34c12-e92c-4a88-9f23-5b684e59de6e admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:43.090298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1286/2578] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:41:42 2022] POST /identity/v3/projects => generated 381 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:41:43.096035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772e92a-916d-47b0-b614-2429eeaecac3 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:43.115851 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772e92a-916d-47b0-b614-2429eeaecac3 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:43.145104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:43.233774 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259b18c-a4cb-4b42-88a9-291de45eee14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:43.234416 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259b18c-a4cb-4b42-88a9-291de45eee14 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:43.234700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259b18c-a4cb-4b42-88a9-291de45eee14 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:43.235002 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259b18c-a4cb-4b42-88a9-291de45eee14 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:43.235684 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259b18c-a4cb-4b42-88a9-291de45eee14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:43.237818 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259b18c-a4cb-4b42-88a9-291de45eee14 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:43.423466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1293/2579] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:41:43 2022] POST /identity/v3/users => generated 297 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:41:43.429043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac34c12-e92c-4a88-9f23-5b684e59de6e admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:43.451778 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac34c12-e92c-4a88-9f23-5b684e59de6e admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:43.463718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc934e8-6fb7-4afd-9a23-c110839a4c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:43.464861 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc934e8-6fb7-4afd-9a23-c110839a4c2e admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:43.465983 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc934e8-6fb7-4afd-9a23-c110839a4c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:43.466468 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc934e8-6fb7-4afd-9a23-c110839a4c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:43.467850 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc934e8-6fb7-4afd-9a23-c110839a4c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:43.470891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc934e8-6fb7-4afd-9a23-c110839a4c2e admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:43.503069 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1287/2580] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:41:43.505055 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259b18c-a4cb-4b42-88a9-291de45eee14 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:43.531339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259b18c-a4cb-4b42-88a9-291de45eee14 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:43.547728 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17693fa-8925-446f-85a3-1df842ee3053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:43.547728 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17693fa-8925-446f-85a3-1df842ee3053 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:43.547728 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17693fa-8925-446f-85a3-1df842ee3053 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:43.548052 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17693fa-8925-446f-85a3-1df842ee3053 admin admin] PATH_INFO: `/v3/projects/56465417fb834219bc2f2f6f7db7509b/users/4c1dc27d3f744dc496d67fff1ec993ee/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:43.580288 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17693fa-8925-446f-85a3-1df842ee3053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56465417fb834219bc2f2f6f7db7509b, user_id=4c1dc27d3f744dc496d67fff1ec993ee, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:43.585638 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17693fa-8925-446f-85a3-1df842ee3053 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:43.767977 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1294/2581] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:41:43 2022] PUT /identity/v3/projects/56465417fb834219bc2f2f6f7db7509b/users/4c1dc27d3f744dc496d67fff1ec993ee/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:43.772970 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc934e8-6fb7-4afd-9a23-c110839a4c2e admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:43.790682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc934e8-6fb7-4afd-9a23-c110839a4c2e admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:43.933474 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7414fab-93ef-47ab-bcde-d38c18763aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:43.934545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7414fab-93ef-47ab-bcde-d38c18763aae admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:43.935093 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7414fab-93ef-47ab-bcde-d38c18763aae admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:43.935601 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7414fab-93ef-47ab-bcde-d38c18763aae admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:43.936854 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7414fab-93ef-47ab-bcde-d38c18763aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:43.940248 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7414fab-93ef-47ab-bcde-d38c18763aae admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:43.962404 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1288/2582] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:41:43.967795 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17693fa-8925-446f-85a3-1df842ee3053 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:43.983729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17693fa-8925-446f-85a3-1df842ee3053 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:43.991913 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f17c5b-4e1e-41b5-bbbf-b597374bdd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:43.993049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f17c5b-4e1e-41b5-bbbf-b597374bdd28 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:43.993448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f17c5b-4e1e-41b5-bbbf-b597374bdd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:43.993844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f17c5b-4e1e-41b5-bbbf-b597374bdd28 admin admin] PATH_INFO: `/v3/projects/56465417fb834219bc2f2f6f7db7509b/users/4c1dc27d3f744dc496d67fff1ec993ee/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:43.997684 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f17c5b-4e1e-41b5-bbbf-b597374bdd28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56465417fb834219bc2f2f6f7db7509b, user_id=4c1dc27d3f744dc496d67fff1ec993ee, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:44.001169 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f17c5b-4e1e-41b5-bbbf-b597374bdd28 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:44.019976 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1295/2583] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:41:43 2022] PUT /identity/v3/projects/56465417fb834219bc2f2f6f7db7509b/users/4c1dc27d3f744dc496d67fff1ec993ee/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:41:44.034216 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed60608-b813-40f6-abd7-eb461a0cb1a8 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:44.034708 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed60608-b813-40f6-abd7-eb461a0cb1a8 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:44.035248 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed60608-b813-40f6-abd7-eb461a0cb1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:44.089058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-0ed60608-b813-40f6-abd7-eb461a0cb1a8 None None] MFA Rules not processed for user `4c1dc27d3f744dc496d67fff1ec993ee`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:41:44.091103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-0ed60608-b813-40f6-abd7-eb461a0cb1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:44.214460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1289/2584] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:41:44 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:41:44.227693 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f17c5b-4e1e-41b5-bbbf-b597374bdd28 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:44.231084 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-55f17c5b-4e1e-41b5-bbbf-b597374bdd28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:44.333070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f17c5b-4e1e-41b5-bbbf-b597374bdd28 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:44.339016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08276d6d-acb1-4474-bdbb-666b3431ac63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:44.339966 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08276d6d-acb1-4474-bdbb-666b3431ac63 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:44.340436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08276d6d-acb1-4474-bdbb-666b3431ac63 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:44.340908 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08276d6d-acb1-4474-bdbb-666b3431ac63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:44.360750 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08276d6d-acb1-4474-bdbb-666b3431ac63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:44.361188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08276d6d-acb1-4474-bdbb-666b3431ac63 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:44.379753 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1296/2585] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:41:44 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:41:44.844244 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed60608-b813-40f6-abd7-eb461a0cb1a8 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:41:44.846200 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-0ed60608-b813-40f6-abd7-eb461a0cb1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:41:44.856780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:44.867976 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed60608-b813-40f6-abd7-eb461a0cb1a8 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:41:44.875326 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6860f9-bd34-4b43-9aed-61ea9c07b029 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:41:44.875902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6860f9-bd34-4b43-9aed-61ea9c07b029 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:41:44.876351 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6860f9-bd34-4b43-9aed-61ea9c07b029 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:41:44.876786 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6860f9-bd34-4b43-9aed-61ea9c07b029 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:41:44.993067 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6860f9-bd34-4b43-9aed-61ea9c07b029 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:41:44.996847 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6860f9-bd34-4b43-9aed-61ea9c07b029 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:41:45.012360 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1290/2586] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:41:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:41:54.371568 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:41:55.143901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=27 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:02.102418 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08276d6d-acb1-4474-bdbb-666b3431ac63 service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:02.253023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08276d6d-acb1-4474-bdbb-666b3431ac63 service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:02.261471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bf6e5-9a3e-4c17-ae59-191d0ad4dfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:02.262670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bf6e5-9a3e-4c17-ae59-191d0ad4dfc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:02.263211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bf6e5-9a3e-4c17-ae59-191d0ad4dfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:02.263854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bf6e5-9a3e-4c17-ae59-191d0ad4dfc5 admin admin] PATH_INFO: `/v3/users/0f1d9342be86475bad8132125a7836d7` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:02.265879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bf6e5-9a3e-4c17-ae59-191d0ad4dfc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f1d9342be86475bad8132125a7836d7)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:02.269810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bf6e5-9a3e-4c17-ae59-191d0ad4dfc5 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:02.643443 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-220bf6e5-9a3e-4c17-ae59-191d0ad4dfc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f1d9342be86475bad8132125a7836d7'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:02.795981 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-220bf6e5-9a3e-4c17-ae59-191d0ad4dfc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f1d9342be86475bad8132125a7836d7'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:02.797526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-220bf6e5-9a3e-4c17-ae59-191d0ad4dfc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f1d9342be86475bad8132125a7836d7'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:02.798899 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-220bf6e5-9a3e-4c17-ae59-191d0ad4dfc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f1d9342be86475bad8132125a7836d7'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:02.816065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-220bf6e5-9a3e-4c17-ae59-191d0ad4dfc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f1d9342be86475bad8132125a7836d7'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:02.858679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1297/2587] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:02 2022] DELETE /identity/v3/users/0f1d9342be86475bad8132125a7836d7 => generated 0 bytes in 758 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:02.866056 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8127c-8f62-4eb4-8592-6fbb5842afbd None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:02.866850 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8127c-8f62-4eb4-8592-6fbb5842afbd None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:02.867719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8127c-8f62-4eb4-8592-6fbb5842afbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:02.909923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-a4f8127c-8f62-4eb4-8592-6fbb5842afbd None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:02.913129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a4f8127c-8f62-4eb4-8592-6fbb5842afbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:03.026213 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1291/2588] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:03.043339 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bf6e5-9a3e-4c17-ae59-191d0ad4dfc5 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:03.045766 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-220bf6e5-9a3e-4c17-ae59-191d0ad4dfc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:03.161356 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bf6e5-9a3e-4c17-ae59-191d0ad4dfc5 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:03.169202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612ed3e-53c0-4685-af8d-7b7beb198973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:03.170219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612ed3e-53c0-4685-af8d-7b7beb198973 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:03.170668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612ed3e-53c0-4685-af8d-7b7beb198973 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:03.171231 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612ed3e-53c0-4685-af8d-7b7beb198973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:03.198314 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612ed3e-53c0-4685-af8d-7b7beb198973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:03.201233 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612ed3e-53c0-4685-af8d-7b7beb198973 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:03.219315 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1298/2589] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:42:03.904011 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8127c-8f62-4eb4-8592-6fbb5842afbd None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:03.906381 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a4f8127c-8f62-4eb4-8592-6fbb5842afbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:03.932173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8127c-8f62-4eb4-8592-6fbb5842afbd None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:03.944519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54338ee5-2f4d-45c0-9190-fed117280ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:03.946584 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54338ee5-2f4d-45c0-9190-fed117280ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:03.948022 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54338ee5-2f4d-45c0-9190-fed117280ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:03.948671 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54338ee5-2f4d-45c0-9190-fed117280ac8 admin admin] PATH_INFO: `/v3/projects/f10e26aa6b684a398ec7452e0356548e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:03.953883 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54338ee5-2f4d-45c0-9190-fed117280ac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f10e26aa6b684a398ec7452e0356548e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:03.976869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54338ee5-2f4d-45c0-9190-fed117280ac8 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:04.158937 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-54338ee5-2f4d-45c0-9190-fed117280ac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f10e26aa6b684a398ec7452e0356548e', 'request_id': 'req-54338ee5-2f4d-45c0-9190-fed117280ac8'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:04.169309 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-54338ee5-2f4d-45c0-9190-fed117280ac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f10e26aa6b684a398ec7452e0356548e', 'request_id': 'req-54338ee5-2f4d-45c0-9190-fed117280ac8'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:04.276722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-54338ee5-2f4d-45c0-9190-fed117280ac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f10e26aa6b684a398ec7452e0356548e', 'request_id': 'req-54338ee5-2f4d-45c0-9190-fed117280ac8'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:04.302677 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-54338ee5-2f4d-45c0-9190-fed117280ac8 admin admin] The token cache is being invalidate because project f10e26aa6b684a398ec7452e0356548e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:04.303233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-54338ee5-2f4d-45c0-9190-fed117280ac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:04.304645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-54338ee5-2f4d-45c0-9190-fed117280ac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:04.309518 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1292/2590] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:03 2022] DELETE /identity/v3/projects/f10e26aa6b684a398ec7452e0356548e => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:04.320857 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612ed3e-53c0-4685-af8d-7b7beb198973 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:04.338307 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612ed3e-53c0-4685-af8d-7b7beb198973 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:04.369971 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:04.436696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81e182-c52b-43e1-b355-fc800ead8821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:04.437982 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81e182-c52b-43e1-b355-fc800ead8821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:04.438700 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81e182-c52b-43e1-b355-fc800ead8821 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:04.439245 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81e182-c52b-43e1-b355-fc800ead8821 admin admin] PATH_INFO: `/v3/users/4c1dc27d3f744dc496d67fff1ec993ee` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:04.446873 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81e182-c52b-43e1-b355-fc800ead8821 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c1dc27d3f744dc496d67fff1ec993ee)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:04.454059 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81e182-c52b-43e1-b355-fc800ead8821 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:05.191139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-af81e182-c52b-43e1-b355-fc800ead8821 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c1dc27d3f744dc496d67fff1ec993ee'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:05.320532 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-af81e182-c52b-43e1-b355-fc800ead8821 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c1dc27d3f744dc496d67fff1ec993ee'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:05.323252 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-af81e182-c52b-43e1-b355-fc800ead8821 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c1dc27d3f744dc496d67fff1ec993ee'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:05.329469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-af81e182-c52b-43e1-b355-fc800ead8821 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c1dc27d3f744dc496d67fff1ec993ee'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:05.336789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-af81e182-c52b-43e1-b355-fc800ead8821 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c1dc27d3f744dc496d67fff1ec993ee'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:05.364071 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1299/2591] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:04 2022] DELETE /identity/v3/users/4c1dc27d3f744dc496d67fff1ec993ee => generated 0 bytes in 1044 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:06.001121 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54338ee5-2f4d-45c0-9190-fed117280ac8 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:06.003414 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-54338ee5-2f4d-45c0-9190-fed117280ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:06.027768 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54338ee5-2f4d-45c0-9190-fed117280ac8 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:06.120440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4148e-1f2d-4c11-b662-91290610fcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:06.121300 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4148e-1f2d-4c11-b662-91290610fcf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:06.121715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4148e-1f2d-4c11-b662-91290610fcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:06.122047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4148e-1f2d-4c11-b662-91290610fcf1 admin admin] PATH_INFO: `/v3/projects/56465417fb834219bc2f2f6f7db7509b` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:06.123578 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4148e-1f2d-4c11-b662-91290610fcf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56465417fb834219bc2f2f6f7db7509b)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:06.125632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4148e-1f2d-4c11-b662-91290610fcf1 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:06.328285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:06.405856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-c1a4148e-1f2d-4c11-b662-91290610fcf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56465417fb834219bc2f2f6f7db7509b', 'request_id': 'req-c1a4148e-1f2d-4c11-b662-91290610fcf1'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:06.405856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-c1a4148e-1f2d-4c11-b662-91290610fcf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56465417fb834219bc2f2f6f7db7509b', 'request_id': 'req-c1a4148e-1f2d-4c11-b662-91290610fcf1'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:06.538723 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-c1a4148e-1f2d-4c11-b662-91290610fcf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56465417fb834219bc2f2f6f7db7509b', 'request_id': 'req-c1a4148e-1f2d-4c11-b662-91290610fcf1'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:06.559734 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-c1a4148e-1f2d-4c11-b662-91290610fcf1 admin admin] The token cache is being invalidate because project 56465417fb834219bc2f2f6f7db7509b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:06.560111 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-c1a4148e-1f2d-4c11-b662-91290610fcf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:06.560802 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-c1a4148e-1f2d-4c11-b662-91290610fcf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:06.562413 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1293/2592] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:05 2022] DELETE /identity/v3/projects/56465417fb834219bc2f2f6f7db7509b => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:06.569756 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dad84-922d-46c9-b27d-4748a1d5297f None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:06.570641 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dad84-922d-46c9-b27d-4748a1d5297f None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:06.570805 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dad84-922d-46c9-b27d-4748a1d5297f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:06.607649 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-8a2dad84-922d-46c9-b27d-4748a1d5297f None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:06.608942 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8a2dad84-922d-46c9-b27d-4748a1d5297f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:06.701381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1300/2593] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:06.716488 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebba319-d676-4fdd-8e45-e4a13f2adcba None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:06.716918 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebba319-d676-4fdd-8e45-e4a13f2adcba None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:06.717382 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebba319-d676-4fdd-8e45-e4a13f2adcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:06.753691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-9ebba319-d676-4fdd-8e45-e4a13f2adcba None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:06.755653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9ebba319-d676-4fdd-8e45-e4a13f2adcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:06.771166 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1294/2594] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:06.777565 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2dad84-922d-46c9-b27d-4748a1d5297f None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:06.795095 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2dad84-922d-46c9-b27d-4748a1d5297f None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:06.803100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5e924-9957-4e04-9667-a4dceb307e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:06.804200 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5e924-9957-4e04-9667-a4dceb307e28 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:06.804655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5e924-9957-4e04-9667-a4dceb307e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:06.805175 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5e924-9957-4e04-9667-a4dceb307e28 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:06.806371 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5e924-9957-4e04-9667-a4dceb307e28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:42:06.806942 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5e924-9957-4e04-9667-a4dceb307e28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:06.810034 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5e924-9957-4e04-9667-a4dceb307e28 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:06.832353 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1301/2595] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:42:06.839428 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebba319-d676-4fdd-8e45-e4a13f2adcba None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:06.850242 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebba319-d676-4fdd-8e45-e4a13f2adcba None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:06.857195 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd302f07-9ff7-4ecd-9cf5-922d6ead5ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:06.858015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd302f07-9ff7-4ecd-9cf5-922d6ead5ebd admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:06.858369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd302f07-9ff7-4ecd-9cf5-922d6ead5ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:06.858649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd302f07-9ff7-4ecd-9cf5-922d6ead5ebd admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:06.859465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd302f07-9ff7-4ecd-9cf5-922d6ead5ebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:06.862434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd302f07-9ff7-4ecd-9cf5-922d6ead5ebd admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:06.966088 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1295/2596] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:06 2022] POST /identity/v3/projects => generated 367 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:06.972252 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5e924-9957-4e04-9667-a4dceb307e28 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:06.996083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5e924-9957-4e04-9667-a4dceb307e28 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:07.049130 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:07.103460 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde64ed6-2492-4bb5-aefe-d35f6f267252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:07.104612 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde64ed6-2492-4bb5-aefe-d35f6f267252 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:07.105767 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde64ed6-2492-4bb5-aefe-d35f6f267252 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:07.105767 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde64ed6-2492-4bb5-aefe-d35f6f267252 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:07.106712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde64ed6-2492-4bb5-aefe-d35f6f267252 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:07.110215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde64ed6-2492-4bb5-aefe-d35f6f267252 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:07.231173 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1302/2597] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:06 2022] POST /identity/v3/users => generated 284 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:07.235406 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd302f07-9ff7-4ecd-9cf5-922d6ead5ebd admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:07.252454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd302f07-9ff7-4ecd-9cf5-922d6ead5ebd admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:07.269068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3043b4-3ef7-4e89-a2de-06c3d82f7c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:07.270125 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3043b4-3ef7-4e89-a2de-06c3d82f7c7d admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:07.270574 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3043b4-3ef7-4e89-a2de-06c3d82f7c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:07.271046 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3043b4-3ef7-4e89-a2de-06c3d82f7c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:07.272205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3043b4-3ef7-4e89-a2de-06c3d82f7c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:07.274570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3043b4-3ef7-4e89-a2de-06c3d82f7c7d admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:07.289515 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1296/2598] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:07.294490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde64ed6-2492-4bb5-aefe-d35f6f267252 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:07.313724 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde64ed6-2492-4bb5-aefe-d35f6f267252 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:07.325983 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae266b-a306-4d23-8042-530550c5e028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:07.326893 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae266b-a306-4d23-8042-530550c5e028 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:07.327379 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae266b-a306-4d23-8042-530550c5e028 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:07.327846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae266b-a306-4d23-8042-530550c5e028 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:07.328969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdae266b-a306-4d23-8042-530550c5e028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:07.331124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdae266b-a306-4d23-8042-530550c5e028 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:07.346152 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1303/2599] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:42:07.354330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3043b4-3ef7-4e89-a2de-06c3d82f7c7d admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:07.372716 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3043b4-3ef7-4e89-a2de-06c3d82f7c7d admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:07.379568 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b9fb9-24af-41a4-80f8-262fed82a505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:07.380394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b9fb9-24af-41a4-80f8-262fed82a505 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:07.380677 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b9fb9-24af-41a4-80f8-262fed82a505 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:07.380956 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b9fb9-24af-41a4-80f8-262fed82a505 admin admin] PATH_INFO: `/v3/projects/e867c73d87ec4254984b2638e62cf741/users/a48161b401c146a7b99b74ca0c569091/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:07.404149 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b9fb9-24af-41a4-80f8-262fed82a505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e867c73d87ec4254984b2638e62cf741, user_id=a48161b401c146a7b99b74ca0c569091, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:07.408282 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b9fb9-24af-41a4-80f8-262fed82a505 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:07.542824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1297/2600] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:07 2022] PUT /identity/v3/projects/e867c73d87ec4254984b2638e62cf741/users/a48161b401c146a7b99b74ca0c569091/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:07.566316 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1c016-bb5f-44f2-bdb7-ef737dcdf23c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:07.566579 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1c016-bb5f-44f2-bdb7-ef737dcdf23c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:07.566897 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1c016-bb5f-44f2-bdb7-ef737dcdf23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:07.614551 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-0bf1c016-bb5f-44f2-bdb7-ef737dcdf23c None None] MFA Rules not processed for user `a48161b401c146a7b99b74ca0c569091`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:07.615875 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0bf1c016-bb5f-44f2-bdb7-ef737dcdf23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:07.858733 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1304/2601] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:07.865366 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b9fb9-24af-41a4-80f8-262fed82a505 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:07.967735 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b9fb9-24af-41a4-80f8-262fed82a505 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:07.975489 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566c16a-f495-484e-aa78-8b83587e1cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:07.976555 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566c16a-f495-484e-aa78-8b83587e1cab admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:07.977035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566c16a-f495-484e-aa78-8b83587e1cab admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:07.977521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566c16a-f495-484e-aa78-8b83587e1cab admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:07.979236 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5566c16a-f495-484e-aa78-8b83587e1cab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:07.982573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5566c16a-f495-484e-aa78-8b83587e1cab admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:08.150182 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1298/2602] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:07 2022] POST /identity/v3/projects => generated 365 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:08.156399 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1c016-bb5f-44f2-bdb7-ef737dcdf23c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:08.176418 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1c016-bb5f-44f2-bdb7-ef737dcdf23c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:08.296042 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fad277-a94b-4459-bf99-9bbf575e22d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:08.297183 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fad277-a94b-4459-bf99-9bbf575e22d1 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:08.297658 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fad277-a94b-4459-bf99-9bbf575e22d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:08.298132 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fad277-a94b-4459-bf99-9bbf575e22d1 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:08.299402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fad277-a94b-4459-bf99-9bbf575e22d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:08.303654 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fad277-a94b-4459-bf99-9bbf575e22d1 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:08.499927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1305/2603] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:08 2022] POST /identity/v3/users => generated 289 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:08.523546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566c16a-f495-484e-aa78-8b83587e1cab admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:08.539131 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566c16a-f495-484e-aa78-8b83587e1cab admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:08.546055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93142d6a-bd18-4b42-8d42-d6a126133883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:08.546764 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93142d6a-bd18-4b42-8d42-d6a126133883 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:08.547281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93142d6a-bd18-4b42-8d42-d6a126133883 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:08.547938 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93142d6a-bd18-4b42-8d42-d6a126133883 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:08.548567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93142d6a-bd18-4b42-8d42-d6a126133883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:08.550625 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93142d6a-bd18-4b42-8d42-d6a126133883 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:08.564793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1299/2604] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:42:08.570490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fad277-a94b-4459-bf99-9bbf575e22d1 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:08.583271 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fad277-a94b-4459-bf99-9bbf575e22d1 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:08.588862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2efb36-de76-4103-a1e5-8ccb0cc9505c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:08.589659 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2efb36-de76-4103-a1e5-8ccb0cc9505c admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:08.589978 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2efb36-de76-4103-a1e5-8ccb0cc9505c admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:08.590293 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2efb36-de76-4103-a1e5-8ccb0cc9505c admin admin] PATH_INFO: `/v3/projects/b425543c07354d9aa32c12fda0b2d932/users/6f7e18ce07344389b49cfdc20cfa5bbf/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:08.618048 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2efb36-de76-4103-a1e5-8ccb0cc9505c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b425543c07354d9aa32c12fda0b2d932, user_id=6f7e18ce07344389b49cfdc20cfa5bbf, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:08.621414 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2efb36-de76-4103-a1e5-8ccb0cc9505c admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:08.922830 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1306/2605] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:08 2022] PUT /identity/v3/projects/b425543c07354d9aa32c12fda0b2d932/users/6f7e18ce07344389b49cfdc20cfa5bbf/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:08.927587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93142d6a-bd18-4b42-8d42-d6a126133883 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:08.940623 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93142d6a-bd18-4b42-8d42-d6a126133883 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:08.976606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:09.042203 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134d4ea-90e3-4df1-b8d2-df5451dbacc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:09.043302 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134d4ea-90e3-4df1-b8d2-df5451dbacc7 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:09.043750 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134d4ea-90e3-4df1-b8d2-df5451dbacc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:09.044196 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134d4ea-90e3-4df1-b8d2-df5451dbacc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:09.045507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134d4ea-90e3-4df1-b8d2-df5451dbacc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:09.048786 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134d4ea-90e3-4df1-b8d2-df5451dbacc7 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:09.073396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1300/2606] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:09.079364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2efb36-de76-4103-a1e5-8ccb0cc9505c admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:09.086863 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:09.096249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2efb36-de76-4103-a1e5-8ccb0cc9505c admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:09.103801 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce25d48-9a9b-4250-aa99-535f915120dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:09.104572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce25d48-9a9b-4250-aa99-535f915120dc admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:09.104856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce25d48-9a9b-4250-aa99-535f915120dc admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:09.105284 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce25d48-9a9b-4250-aa99-535f915120dc admin admin] PATH_INFO: `/v3/projects/b425543c07354d9aa32c12fda0b2d932/users/6f7e18ce07344389b49cfdc20cfa5bbf/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:09.107469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce25d48-9a9b-4250-aa99-535f915120dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b425543c07354d9aa32c12fda0b2d932, user_id=6f7e18ce07344389b49cfdc20cfa5bbf, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:09.110192 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce25d48-9a9b-4250-aa99-535f915120dc admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:09.123721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1307/2607] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:09 2022] PUT /identity/v3/projects/b425543c07354d9aa32c12fda0b2d932/users/6f7e18ce07344389b49cfdc20cfa5bbf/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:09.136258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6556a1f-a4b7-4694-a60a-194a42cfe80c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:09.136541 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6556a1f-a4b7-4694-a60a-194a42cfe80c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:09.136818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6556a1f-a4b7-4694-a60a-194a42cfe80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:09.174597 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-a6556a1f-a4b7-4694-a60a-194a42cfe80c None None] MFA Rules not processed for user `6f7e18ce07344389b49cfdc20cfa5bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:09.176170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a6556a1f-a4b7-4694-a60a-194a42cfe80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:09.301731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1301/2608] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:42:09.317189 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce25d48-9a9b-4250-aa99-535f915120dc admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:09.319792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7ce25d48-9a9b-4250-aa99-535f915120dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:09.434573 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce25d48-9a9b-4250-aa99-535f915120dc admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:09.440925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70e5a1-c9bf-4909-b102-ebffd9871480 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:09.441657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70e5a1-c9bf-4909-b102-ebffd9871480 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:09.442090 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70e5a1-c9bf-4909-b102-ebffd9871480 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:09.442455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70e5a1-c9bf-4909-b102-ebffd9871480 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:09.459442 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70e5a1-c9bf-4909-b102-ebffd9871480 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:09.461374 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70e5a1-c9bf-4909-b102-ebffd9871480 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:09.484697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1308/2609] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:42:14.311822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6556a1f-a4b7-4694-a60a-194a42cfe80c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:14.445745 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6556a1f-a4b7-4694-a60a-194a42cfe80c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:14.455429 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d1af3-9689-41f2-be31-058fe7cab700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:14.456210 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d1af3-9689-41f2-be31-058fe7cab700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:14.456515 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d1af3-9689-41f2-be31-058fe7cab700 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:14.456834 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d1af3-9689-41f2-be31-058fe7cab700 admin admin] PATH_INFO: `/v3/users/a48161b401c146a7b99b74ca0c569091` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:14.458335 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d1af3-9689-41f2-be31-058fe7cab700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a48161b401c146a7b99b74ca0c569091)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:14.461365 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d1af3-9689-41f2-be31-058fe7cab700 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:14.802535 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-432d1af3-9689-41f2-be31-058fe7cab700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a48161b401c146a7b99b74ca0c569091'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:14.805346 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:14.831638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-432d1af3-9689-41f2-be31-058fe7cab700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a48161b401c146a7b99b74ca0c569091'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:14.833747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-432d1af3-9689-41f2-be31-058fe7cab700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a48161b401c146a7b99b74ca0c569091'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:14.847723 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-432d1af3-9689-41f2-be31-058fe7cab700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a48161b401c146a7b99b74ca0c569091'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:14.848747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-432d1af3-9689-41f2-be31-058fe7cab700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a48161b401c146a7b99b74ca0c569091'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:14.945054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1302/2610] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:14 2022] DELETE /identity/v3/users/a48161b401c146a7b99b74ca0c569091 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:14.953882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c747a38-03c8-4dd8-8143-e6dedd17262e None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:14.955963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c747a38-03c8-4dd8-8143-e6dedd17262e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:14.955963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c747a38-03c8-4dd8-8143-e6dedd17262e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:14.997817 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-5c747a38-03c8-4dd8-8143-e6dedd17262e None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:14.999960 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-5c747a38-03c8-4dd8-8143-e6dedd17262e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:15.135035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1309/2611] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:15.156764 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d1af3-9689-41f2-be31-058fe7cab700 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:15.160565 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-432d1af3-9689-41f2-be31-058fe7cab700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:15.245128 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c747a38-03c8-4dd8-8143-e6dedd17262e None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:15.249134 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-5c747a38-03c8-4dd8-8143-e6dedd17262e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:15.275325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c747a38-03c8-4dd8-8143-e6dedd17262e None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:15.288651 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa45eb4-0974-4562-a04b-12d12e26d7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:15.290857 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa45eb4-0974-4562-a04b-12d12e26d7af admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:15.291485 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa45eb4-0974-4562-a04b-12d12e26d7af admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:15.291997 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa45eb4-0974-4562-a04b-12d12e26d7af admin admin] PATH_INFO: `/v3/users/4bd8a483dc6147258af1681ec0e8e4a3` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:15.301687 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa45eb4-0974-4562-a04b-12d12e26d7af admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bd8a483dc6147258af1681ec0e8e4a3)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:15.310509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa45eb4-0974-4562-a04b-12d12e26d7af admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:15.372082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d1af3-9689-41f2-be31-058fe7cab700 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:15.389579 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204b561-9c58-4c84-a840-d472b853de85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:15.390776 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204b561-9c58-4c84-a840-d472b853de85 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:15.391662 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204b561-9c58-4c84-a840-d472b853de85 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:15.392176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204b561-9c58-4c84-a840-d472b853de85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:15.418627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204b561-9c58-4c84-a840-d472b853de85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:15.421636 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204b561-9c58-4c84-a840-d472b853de85 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:15.458052 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1303/2612] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:42:15.601846 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5aa45eb4-0974-4562-a04b-12d12e26d7af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bd8a483dc6147258af1681ec0e8e4a3'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:15.734049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5aa45eb4-0974-4562-a04b-12d12e26d7af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bd8a483dc6147258af1681ec0e8e4a3'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:15.739586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5aa45eb4-0974-4562-a04b-12d12e26d7af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bd8a483dc6147258af1681ec0e8e4a3'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:15.739586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5aa45eb4-0974-4562-a04b-12d12e26d7af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bd8a483dc6147258af1681ec0e8e4a3'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:15.753595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5aa45eb4-0974-4562-a04b-12d12e26d7af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bd8a483dc6147258af1681ec0e8e4a3'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:15.803883 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1310/2613] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:15 2022] DELETE /identity/v3/users/4bd8a483dc6147258af1681ec0e8e4a3 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:15.814958 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2fcea-337e-4f82-87a6-6ae29f7b9156 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:15.815828 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2fcea-337e-4f82-87a6-6ae29f7b9156 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:15.816295 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2fcea-337e-4f82-87a6-6ae29f7b9156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:15.857157 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-feb2fcea-337e-4f82-87a6-6ae29f7b9156 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:15.859774 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-feb2fcea-337e-4f82-87a6-6ae29f7b9156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:15.990744 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1304/2614] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:16.014216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa45eb4-0974-4562-a04b-12d12e26d7af admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:16.018349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-5aa45eb4-0974-4562-a04b-12d12e26d7af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:16.033653 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:16.159036 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa45eb4-0974-4562-a04b-12d12e26d7af admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:16.165646 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3cd8f-3703-4fa4-a46c-872fefd7f9c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:16.166720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3cd8f-3703-4fa4-a46c-872fefd7f9c7 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:16.167228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3cd8f-3703-4fa4-a46c-872fefd7f9c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:16.167748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3cd8f-3703-4fa4-a46c-872fefd7f9c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:16.186629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3cd8f-3703-4fa4-a46c-872fefd7f9c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:16.189288 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3cd8f-3703-4fa4-a46c-872fefd7f9c7 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:16.203884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2fcea-337e-4f82-87a6-6ae29f7b9156 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:16.208691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-feb2fcea-337e-4f82-87a6-6ae29f7b9156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:16.213196 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1311/2615] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:42:16.256240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2fcea-337e-4f82-87a6-6ae29f7b9156 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:16.275730 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:16.277427 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:16.277427 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:16.279209 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1 admin admin] PATH_INFO: `/v3/projects/e867c73d87ec4254984b2638e62cf741` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:16.282766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e867c73d87ec4254984b2638e62cf741)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:16.286739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:16.466722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e867c73d87ec4254984b2638e62cf741', 'request_id': 'req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:16.480333 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e867c73d87ec4254984b2638e62cf741', 'request_id': 'req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:16.589880 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e867c73d87ec4254984b2638e62cf741', 'request_id': 'req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:16.603222 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:16.614735 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1 admin admin] The token cache is being invalidate because project e867c73d87ec4254984b2638e62cf741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:16.615429 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:16.616655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:16.620556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1305/2616] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:16 2022] DELETE /identity/v3/projects/e867c73d87ec4254984b2638e62cf741 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:16.632141 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3cd8f-3703-4fa4-a46c-872fefd7f9c7 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:16.648777 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3cd8f-3703-4fa4-a46c-872fefd7f9c7 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:16.806707 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a4dc4-5c42-4629-91db-7ae8e80b8828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:16.807960 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a4dc4-5c42-4629-91db-7ae8e80b8828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:16.808547 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a4dc4-5c42-4629-91db-7ae8e80b8828 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:16.809161 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a4dc4-5c42-4629-91db-7ae8e80b8828 admin admin] PATH_INFO: `/v3/users/6f7e18ce07344389b49cfdc20cfa5bbf` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:16.812976 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a4dc4-5c42-4629-91db-7ae8e80b8828 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f7e18ce07344389b49cfdc20cfa5bbf)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:16.818764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a4dc4-5c42-4629-91db-7ae8e80b8828 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:17.135839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:17.138134 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:17.154936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-934a4dc4-5c42-4629-91db-7ae8e80b8828 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f7e18ce07344389b49cfdc20cfa5bbf'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:17.190078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3ceb6-820d-4af2-bf05-efe559ee2fa1 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:17.220559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92889a-e94e-416c-b8a8-d3d1d26731a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:17.221161 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92889a-e94e-416c-b8a8-d3d1d26731a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:17.222820 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92889a-e94e-416c-b8a8-d3d1d26731a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:17.223346 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92889a-e94e-416c-b8a8-d3d1d26731a8 admin admin] PATH_INFO: `/v3/projects/b22f84a7fdc24440b22a2e0ba9fb11e1` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:17.226326 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92889a-e94e-416c-b8a8-d3d1d26731a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b22f84a7fdc24440b22a2e0ba9fb11e1)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:17.230102 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92889a-e94e-416c-b8a8-d3d1d26731a8 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:17.355464 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-934a4dc4-5c42-4629-91db-7ae8e80b8828 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f7e18ce07344389b49cfdc20cfa5bbf'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:17.356759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-934a4dc4-5c42-4629-91db-7ae8e80b8828 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f7e18ce07344389b49cfdc20cfa5bbf'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:17.359751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-934a4dc4-5c42-4629-91db-7ae8e80b8828 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f7e18ce07344389b49cfdc20cfa5bbf'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:17.375975 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-934a4dc4-5c42-4629-91db-7ae8e80b8828 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f7e18ce07344389b49cfdc20cfa5bbf'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:17.412628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1312/2617] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:16 2022] DELETE /identity/v3/users/6f7e18ce07344389b49cfdc20cfa5bbf => generated 0 bytes in 786 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:17.442471 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ad92889a-e94e-416c-b8a8-d3d1d26731a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b22f84a7fdc24440b22a2e0ba9fb11e1', 'request_id': 'req-ad92889a-e94e-416c-b8a8-d3d1d26731a8'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:17.455879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ad92889a-e94e-416c-b8a8-d3d1d26731a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b22f84a7fdc24440b22a2e0ba9fb11e1', 'request_id': 'req-ad92889a-e94e-416c-b8a8-d3d1d26731a8'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:17.612091 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ad92889a-e94e-416c-b8a8-d3d1d26731a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b22f84a7fdc24440b22a2e0ba9fb11e1', 'request_id': 'req-ad92889a-e94e-416c-b8a8-d3d1d26731a8'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:17.631126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ad92889a-e94e-416c-b8a8-d3d1d26731a8 admin admin] The token cache is being invalidate because project b22f84a7fdc24440b22a2e0ba9fb11e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:17.631586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ad92889a-e94e-416c-b8a8-d3d1d26731a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:17.632585 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ad92889a-e94e-416c-b8a8-d3d1d26731a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:17.635975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1306/2618] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:17 2022] DELETE /identity/v3/projects/b22f84a7fdc24440b22a2e0ba9fb11e1 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:17.641568 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a4dc4-5c42-4629-91db-7ae8e80b8828 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:17.644002 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-934a4dc4-5c42-4629-91db-7ae8e80b8828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:17.699757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:17.760548 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a4dc4-5c42-4629-91db-7ae8e80b8828 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:17.904083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f03747c-6f2c-4b8a-bfbd-ccbe7d61273e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:17.905221 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03747c-6f2c-4b8a-bfbd-ccbe7d61273e admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:17.905692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03747c-6f2c-4b8a-bfbd-ccbe7d61273e admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:17.906192 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f03747c-6f2c-4b8a-bfbd-ccbe7d61273e admin admin] PATH_INFO: `/v3/users/9cdbabe1186744c3bd3139bb767d1f30` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:17.908796 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f03747c-6f2c-4b8a-bfbd-ccbe7d61273e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cdbabe1186744c3bd3139bb767d1f30)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:17.912644 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f03747c-6f2c-4b8a-bfbd-ccbe7d61273e admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:18.209907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92889a-e94e-416c-b8a8-d3d1d26731a8 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:18.215184 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ad92889a-e94e-416c-b8a8-d3d1d26731a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:18.242239 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-1f03747c-6f2c-4b8a-bfbd-ccbe7d61273e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cdbabe1186744c3bd3139bb767d1f30'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:18.270492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92889a-e94e-416c-b8a8-d3d1d26731a8 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:18.283015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722d369-c7ee-4ea1-96a0-3e4857761115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:18.284804 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722d369-c7ee-4ea1-96a0-3e4857761115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:18.285419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722d369-c7ee-4ea1-96a0-3e4857761115 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:18.285965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722d369-c7ee-4ea1-96a0-3e4857761115 admin admin] PATH_INFO: `/v3/projects/b425543c07354d9aa32c12fda0b2d932` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:18.288325 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4722d369-c7ee-4ea1-96a0-3e4857761115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b425543c07354d9aa32c12fda0b2d932)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:18.293997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4722d369-c7ee-4ea1-96a0-3e4857761115 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:18.372591 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-1f03747c-6f2c-4b8a-bfbd-ccbe7d61273e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cdbabe1186744c3bd3139bb767d1f30'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:18.373881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-1f03747c-6f2c-4b8a-bfbd-ccbe7d61273e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cdbabe1186744c3bd3139bb767d1f30'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:18.377684 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-1f03747c-6f2c-4b8a-bfbd-ccbe7d61273e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cdbabe1186744c3bd3139bb767d1f30'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:18.396094 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-1f03747c-6f2c-4b8a-bfbd-ccbe7d61273e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cdbabe1186744c3bd3139bb767d1f30'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:18.436355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1313/2619] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:17 2022] DELETE /identity/v3/users/9cdbabe1186744c3bd3139bb767d1f30 => generated 0 bytes in 796 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:18.571414 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-4722d369-c7ee-4ea1-96a0-3e4857761115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b425543c07354d9aa32c12fda0b2d932', 'request_id': 'req-4722d369-c7ee-4ea1-96a0-3e4857761115'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:18.583165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-4722d369-c7ee-4ea1-96a0-3e4857761115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b425543c07354d9aa32c12fda0b2d932', 'request_id': 'req-4722d369-c7ee-4ea1-96a0-3e4857761115'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:18.703819 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-4722d369-c7ee-4ea1-96a0-3e4857761115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b425543c07354d9aa32c12fda0b2d932', 'request_id': 'req-4722d369-c7ee-4ea1-96a0-3e4857761115'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:18.732610 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-4722d369-c7ee-4ea1-96a0-3e4857761115 admin admin] The token cache is being invalidate because project b425543c07354d9aa32c12fda0b2d932 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:18.733215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-4722d369-c7ee-4ea1-96a0-3e4857761115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:18.734455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-4722d369-c7ee-4ea1-96a0-3e4857761115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:18.737163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1307/2620] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:18 2022] DELETE /identity/v3/projects/b425543c07354d9aa32c12fda0b2d932 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:18.745985 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d46082-1c5b-4152-9b68-f87059e0b93e None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:18.747303 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d46082-1c5b-4152-9b68-f87059e0b93e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:18.748602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d46082-1c5b-4152-9b68-f87059e0b93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:18.785312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-66d46082-1c5b-4152-9b68-f87059e0b93e None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:18.788197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-66d46082-1c5b-4152-9b68-f87059e0b93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:18.929415 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1314/2621] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:18.942168 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67fbbe-11eb-48eb-bb3e-bcb5b8f17bad None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:18.942473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67fbbe-11eb-48eb-bb3e-bcb5b8f17bad None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:18.943338 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67fbbe-11eb-48eb-bb3e-bcb5b8f17bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:18.988916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-fb67fbbe-11eb-48eb-bb3e-bcb5b8f17bad None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:18.991102 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-fb67fbbe-11eb-48eb-bb3e-bcb5b8f17bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:19.010412 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1308/2622] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:19.012023 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d46082-1c5b-4152-9b68-f87059e0b93e None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:19.030040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d46082-1c5b-4152-9b68-f87059e0b93e None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:19.036583 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba031dde-11ac-45e8-b85f-c1e734e2b565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:19.037369 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba031dde-11ac-45e8-b85f-c1e734e2b565 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:19.037848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba031dde-11ac-45e8-b85f-c1e734e2b565 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:19.038224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba031dde-11ac-45e8-b85f-c1e734e2b565 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:19.039225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba031dde-11ac-45e8-b85f-c1e734e2b565 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:42:19.039809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba031dde-11ac-45e8-b85f-c1e734e2b565 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:19.042181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba031dde-11ac-45e8-b85f-c1e734e2b565 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:19.064567 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1315/2623] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:42:19.072698 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67fbbe-11eb-48eb-bb3e-bcb5b8f17bad None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:19.090239 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67fbbe-11eb-48eb-bb3e-bcb5b8f17bad None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:19.097985 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6a0de-230f-4a9e-84bd-e16b045e8880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:19.098789 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6a0de-230f-4a9e-84bd-e16b045e8880 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:19.099185 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6a0de-230f-4a9e-84bd-e16b045e8880 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:19.099495 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6a0de-230f-4a9e-84bd-e16b045e8880 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:19.100157 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6a0de-230f-4a9e-84bd-e16b045e8880 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:19.102163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6a0de-230f-4a9e-84bd-e16b045e8880 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:19.173061 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:19.216839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba031dde-11ac-45e8-b85f-c1e734e2b565 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:19.218514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1309/2624] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:19 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:42:19.223982 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6a0de-230f-4a9e-84bd-e16b045e8880 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:19.224259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ba031dde-11ac-45e8-b85f-c1e734e2b565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:19.252566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba031dde-11ac-45e8-b85f-c1e734e2b565 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:19.293777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:19.361973 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6a0de-230f-4a9e-84bd-e16b045e8880 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:19.370828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743f3df-c234-492a-983d-c4ee9b947d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:19.372228 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743f3df-c234-492a-983d-c4ee9b947d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:19.372545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c0814-7dc5-48d6-8ae7-8cb63f3f6da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:19.372845 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743f3df-c234-492a-983d-c4ee9b947d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:19.373275 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743f3df-c234-492a-983d-c4ee9b947d06 admin admin] PATH_INFO: `/v3/projects/d5192e41fd7440db952fae68a130e66e` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:19.373790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c0814-7dc5-48d6-8ae7-8cb63f3f6da1 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:19.374279 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c0814-7dc5-48d6-8ae7-8cb63f3f6da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:19.375056 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c0814-7dc5-48d6-8ae7-8cb63f3f6da1 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:19.376034 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743f3df-c234-492a-983d-c4ee9b947d06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5192e41fd7440db952fae68a130e66e)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:19.376259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c0814-7dc5-48d6-8ae7-8cb63f3f6da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:19.379057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743f3df-c234-492a-983d-c4ee9b947d06 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:19.380559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c0814-7dc5-48d6-8ae7-8cb63f3f6da1 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:19.547442 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1310/2625] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:19 2022] POST /identity/v3/users => generated 291 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:42:19.554129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c0814-7dc5-48d6-8ae7-8cb63f3f6da1 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:19.569190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c0814-7dc5-48d6-8ae7-8cb63f3f6da1 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:19.576215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093bdfc-2939-4909-9cbe-f2e178e54af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:19.577325 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093bdfc-2939-4909-9cbe-f2e178e54af9 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:19.577769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093bdfc-2939-4909-9cbe-f2e178e54af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:19.578227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093bdfc-2939-4909-9cbe-f2e178e54af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:19.579194 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093bdfc-2939-4909-9cbe-f2e178e54af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:19.581681 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093bdfc-2939-4909-9cbe-f2e178e54af9 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:19.597447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1311/2626] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:42:19.604908 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093bdfc-2939-4909-9cbe-f2e178e54af9 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:19.620159 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093bdfc-2939-4909-9cbe-f2e178e54af9 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:19.626353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4167ef-0b48-4302-8d21-ab5f6d4b3466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:19.627240 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4167ef-0b48-4302-8d21-ab5f6d4b3466 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:19.627699 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4167ef-0b48-4302-8d21-ab5f6d4b3466 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:19.628052 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4167ef-0b48-4302-8d21-ab5f6d4b3466 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:19.629008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4167ef-0b48-4302-8d21-ab5f6d4b3466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:19.631879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4167ef-0b48-4302-8d21-ab5f6d4b3466 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:19.657642 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1312/2627] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:42:19.666692 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4167ef-0b48-4302-8d21-ab5f6d4b3466 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:19.689713 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4167ef-0b48-4302-8d21-ab5f6d4b3466 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:19.712722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b59e53-be68-49b1-9554-eb766f3dd0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:19.714098 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b59e53-be68-49b1-9554-eb766f3dd0d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:19.714288 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b59e53-be68-49b1-9554-eb766f3dd0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:19.714755 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b59e53-be68-49b1-9554-eb766f3dd0d9 admin admin] PATH_INFO: `/v3/projects/bf2003df716e424f888324b8227da9f8/users/c9bb9aecd24c4a798e8f48a3e93040cb/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:19.747632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b59e53-be68-49b1-9554-eb766f3dd0d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf2003df716e424f888324b8227da9f8, user_id=c9bb9aecd24c4a798e8f48a3e93040cb, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:19.753329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b59e53-be68-49b1-9554-eb766f3dd0d9 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:20.033682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1313/2628] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:19 2022] PUT /identity/v3/projects/bf2003df716e424f888324b8227da9f8/users/c9bb9aecd24c4a798e8f48a3e93040cb/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:20.049249 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa836d-62b6-43f7-971a-480ddcd9a8c5 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:20.049754 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa836d-62b6-43f7-971a-480ddcd9a8c5 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:20.050202 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa836d-62b6-43f7-971a-480ddcd9a8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:20.057843 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a743f3df-c234-492a-983d-c4ee9b947d06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5192e41fd7440db952fae68a130e66e', 'request_id': 'req-a743f3df-c234-492a-983d-c4ee9b947d06'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:20.068556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a743f3df-c234-492a-983d-c4ee9b947d06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5192e41fd7440db952fae68a130e66e', 'request_id': 'req-a743f3df-c234-492a-983d-c4ee9b947d06'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:20.194900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a743f3df-c234-492a-983d-c4ee9b947d06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5192e41fd7440db952fae68a130e66e', 'request_id': 'req-a743f3df-c234-492a-983d-c4ee9b947d06'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:20.217709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a743f3df-c234-492a-983d-c4ee9b947d06 admin admin] The token cache is being invalidate because project d5192e41fd7440db952fae68a130e66e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:20.218229 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a743f3df-c234-492a-983d-c4ee9b947d06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:20.219517 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a743f3df-c234-492a-983d-c4ee9b947d06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:20.222809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1316/2629] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:19 2022] DELETE /identity/v3/projects/d5192e41fd7440db952fae68a130e66e => generated 0 bytes in 1006 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:20.232527 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659c828-8a52-47b8-9bfa-f5950b47de70 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:20.232934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659c828-8a52-47b8-9bfa-f5950b47de70 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:20.233526 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659c828-8a52-47b8-9bfa-f5950b47de70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:20.269664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-7659c828-8a52-47b8-9bfa-f5950b47de70 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:20.272057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7659c828-8a52-47b8-9bfa-f5950b47de70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:20.283871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-88fa836d-62b6-43f7-971a-480ddcd9a8c5 None None] MFA Rules not processed for user `c9bb9aecd24c4a798e8f48a3e93040cb`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:20.287263 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-88fa836d-62b6-43f7-971a-480ddcd9a8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:20.388326 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1317/2630] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:42:20.402856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7799b4b-26bd-432a-9d54-075ac0e75028 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:20.403446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7799b4b-26bd-432a-9d54-075ac0e75028 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:20.403957 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7799b4b-26bd-432a-9d54-075ac0e75028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:20.412648 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1314/2631] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:42:20.420074 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa836d-62b6-43f7-971a-480ddcd9a8c5 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:20.422359 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-88fa836d-62b6-43f7-971a-480ddcd9a8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:20.437429 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-f7799b4b-26bd-432a-9d54-075ac0e75028 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:20.440039 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f7799b4b-26bd-432a-9d54-075ac0e75028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:20.447941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa836d-62b6-43f7-971a-480ddcd9a8c5 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:20.454016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007360d-c438-480d-be79-caf789bbed63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:20.455003 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1318/2632] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:20.455196 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007360d-c438-480d-be79-caf789bbed63 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:20.455480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007360d-c438-480d-be79-caf789bbed63 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:20.455941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007360d-c438-480d-be79-caf789bbed63 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:20.456844 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007360d-c438-480d-be79-caf789bbed63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:20.460972 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007360d-c438-480d-be79-caf789bbed63 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:20.462830 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7799b4b-26bd-432a-9d54-075ac0e75028 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:20.479571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7799b4b-26bd-432a-9d54-075ac0e75028 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:20.487744 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd5152-a001-46e9-9b4f-98673cc9dd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:20.489035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd5152-a001-46e9-9b4f-98673cc9dd9e admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:20.489317 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd5152-a001-46e9-9b4f-98673cc9dd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:20.489827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd5152-a001-46e9-9b4f-98673cc9dd9e admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:20.491718 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd5152-a001-46e9-9b4f-98673cc9dd9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:42:20.492729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd5152-a001-46e9-9b4f-98673cc9dd9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:20.496212 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:20.499128 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd5152-a001-46e9-9b4f-98673cc9dd9e admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:20.517726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1319/2633] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:42:20.523904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd5152-a001-46e9-9b4f-98673cc9dd9e admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:20.539042 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd5152-a001-46e9-9b4f-98673cc9dd9e admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:20.549956 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311b4ef-aca2-4e62-8965-fd7158a776dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:20.550707 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311b4ef-aca2-4e62-8965-fd7158a776dd admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:20.551237 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311b4ef-aca2-4e62-8965-fd7158a776dd admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:20.560269 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311b4ef-aca2-4e62-8965-fd7158a776dd admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:20.560269 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311b4ef-aca2-4e62-8965-fd7158a776dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:20.560269 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311b4ef-aca2-4e62-8965-fd7158a776dd admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:20.568057 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1315/2634] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:20 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:20.569995 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007360d-c438-480d-be79-caf789bbed63 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:20.642484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:20.679892 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007360d-c438-480d-be79-caf789bbed63 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:20.687968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704599d-ee50-4d88-b3ca-acb1e76787bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:20.689006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704599d-ee50-4d88-b3ca-acb1e76787bb admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:20.689506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704599d-ee50-4d88-b3ca-acb1e76787bb admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:20.690130 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704599d-ee50-4d88-b3ca-acb1e76787bb admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:20.691571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f704599d-ee50-4d88-b3ca-acb1e76787bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:20.698004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1320/2635] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:20 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:42:20.698337 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f704599d-ee50-4d88-b3ca-acb1e76787bb admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:20.700841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311b4ef-aca2-4e62-8965-fd7158a776dd admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:20.804215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1316/2636] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:20 2022] POST /identity/v3/users => generated 298 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:20.812186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704599d-ee50-4d88-b3ca-acb1e76787bb admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:20.826243 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311b4ef-aca2-4e62-8965-fd7158a776dd admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:20.832254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704599d-ee50-4d88-b3ca-acb1e76787bb admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:20.839074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02de4ef-ba95-49ac-a102-aee89287c8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:20.839074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02de4ef-ba95-49ac-a102-aee89287c8a2 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:20.839074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02de4ef-ba95-49ac-a102-aee89287c8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:20.839074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02de4ef-ba95-49ac-a102-aee89287c8a2 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:20.840090 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02de4ef-ba95-49ac-a102-aee89287c8a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:20.841894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db5e3c-d5af-4e2a-897c-28c824c4c6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:20.843302 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db5e3c-d5af-4e2a-897c-28c824c4c6af admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:20.844063 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db5e3c-d5af-4e2a-897c-28c824c4c6af admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:20.844211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02de4ef-ba95-49ac-a102-aee89287c8a2 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:20.844503 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db5e3c-d5af-4e2a-897c-28c824c4c6af admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:20.846052 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db5e3c-d5af-4e2a-897c-28c824c4c6af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:20.849746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db5e3c-d5af-4e2a-897c-28c824c4c6af admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:20.871848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1317/2637] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:42:20.878479 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db5e3c-d5af-4e2a-897c-28c824c4c6af admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:20.896050 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db5e3c-d5af-4e2a-897c-28c824c4c6af admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:20.902946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1ca4c-b8e3-4da8-8c0b-b09bb945ddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:20.907773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1ca4c-b8e3-4da8-8c0b-b09bb945ddf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:20.907773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1ca4c-b8e3-4da8-8c0b-b09bb945ddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:20.907773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1ca4c-b8e3-4da8-8c0b-b09bb945ddf1 admin admin] PATH_INFO: `/v3/projects/255a71c8d92448f2910bbfbb026df440/users/55801f07dc1f45d684bf46a19cdb1fad/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:20.931574 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1ca4c-b8e3-4da8-8c0b-b09bb945ddf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=255a71c8d92448f2910bbfbb026df440, user_id=55801f07dc1f45d684bf46a19cdb1fad, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:20.937076 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1ca4c-b8e3-4da8-8c0b-b09bb945ddf1 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:20.975529 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1321/2638] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:20 2022] POST /identity/v3/users => generated 293 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:42:20.983557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02de4ef-ba95-49ac-a102-aee89287c8a2 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:21.004354 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02de4ef-ba95-49ac-a102-aee89287c8a2 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:21.013473 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47706eb8-a2e0-48ec-a0fe-7738111b380f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:21.014546 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47706eb8-a2e0-48ec-a0fe-7738111b380f admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:21.015045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47706eb8-a2e0-48ec-a0fe-7738111b380f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:21.015501 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47706eb8-a2e0-48ec-a0fe-7738111b380f admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:21.016791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47706eb8-a2e0-48ec-a0fe-7738111b380f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:21.019853 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47706eb8-a2e0-48ec-a0fe-7738111b380f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:21.043065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1322/2639] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:21.052883 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47706eb8-a2e0-48ec-a0fe-7738111b380f admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:21.073543 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47706eb8-a2e0-48ec-a0fe-7738111b380f admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:21.082387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2aaef-a256-4623-ba56-d5ed71b2af01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:21.083368 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2aaef-a256-4623-ba56-d5ed71b2af01 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:21.084400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2aaef-a256-4623-ba56-d5ed71b2af01 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:21.085999 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2aaef-a256-4623-ba56-d5ed71b2af01 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:21.085999 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2aaef-a256-4623-ba56-d5ed71b2af01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:21.088850 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2aaef-a256-4623-ba56-d5ed71b2af01 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:21.113436 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1318/2640] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:20 2022] PUT /identity/v3/projects/255a71c8d92448f2910bbfbb026df440/users/55801f07dc1f45d684bf46a19cdb1fad/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:21.113668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1323/2641] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:21.115643 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2aaef-a256-4623-ba56-d5ed71b2af01 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:21.116213 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1ca4c-b8e3-4da8-8c0b-b09bb945ddf1 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:21.137746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2aaef-a256-4623-ba56-d5ed71b2af01 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:21.265904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3083b-d471-4e65-9b09-5f5cefdb24dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:21.266647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1ca4c-b8e3-4da8-8c0b-b09bb945ddf1 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:21.267701 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3083b-d471-4e65-9b09-5f5cefdb24dd admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:21.268359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3083b-d471-4e65-9b09-5f5cefdb24dd admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:21.268996 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3083b-d471-4e65-9b09-5f5cefdb24dd admin admin] PATH_INFO: `/v3/projects/f2068ee0e18a44e0b3e4d568b7d8ed15/users/3f6d956add8045889ca198926b4eb7af/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:21.276756 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238551d-de7c-439c-8f7f-3a82f336e80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:21.277674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238551d-de7c-439c-8f7f-3a82f336e80f admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:21.278043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238551d-de7c-439c-8f7f-3a82f336e80f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:21.278442 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238551d-de7c-439c-8f7f-3a82f336e80f admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:21.279482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5238551d-de7c-439c-8f7f-3a82f336e80f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:21.282945 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5238551d-de7c-439c-8f7f-3a82f336e80f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:21.296230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3083b-d471-4e65-9b09-5f5cefdb24dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2068ee0e18a44e0b3e4d568b7d8ed15, user_id=3f6d956add8045889ca198926b4eb7af, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:21.304437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3083b-d471-4e65-9b09-5f5cefdb24dd admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:21.314275 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1319/2642] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:21.321654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238551d-de7c-439c-8f7f-3a82f336e80f admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:21.341256 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238551d-de7c-439c-8f7f-3a82f336e80f admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:21.349451 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0ab7b-d07f-4597-8d58-b995985004d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:21.350149 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0ab7b-d07f-4597-8d58-b995985004d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:21.350426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0ab7b-d07f-4597-8d58-b995985004d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:21.350722 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0ab7b-d07f-4597-8d58-b995985004d9 admin admin] PATH_INFO: `/v3/projects/255a71c8d92448f2910bbfbb026df440/users/55801f07dc1f45d684bf46a19cdb1fad/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:21.354082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0ab7b-d07f-4597-8d58-b995985004d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=255a71c8d92448f2910bbfbb026df440, user_id=55801f07dc1f45d684bf46a19cdb1fad, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:21.359245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0ab7b-d07f-4597-8d58-b995985004d9 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:21.387122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1320/2643] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:21 2022] PUT /identity/v3/projects/255a71c8d92448f2910bbfbb026df440/users/55801f07dc1f45d684bf46a19cdb1fad/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:21.394088 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60138248-7200-45dd-9917-2d2cabc03e7c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:21.394737 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60138248-7200-45dd-9917-2d2cabc03e7c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:21.395443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60138248-7200-45dd-9917-2d2cabc03e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:21.441693 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1324/2644] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:21 2022] PUT /identity/v3/projects/f2068ee0e18a44e0b3e4d568b7d8ed15/users/3f6d956add8045889ca198926b4eb7af/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:21.449496 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-60138248-7200-45dd-9917-2d2cabc03e7c None None] MFA Rules not processed for user `55801f07dc1f45d684bf46a19cdb1fad`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:21.451676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-60138248-7200-45dd-9917-2d2cabc03e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:21.466737 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62196775-7a4c-42eb-8b44-3847d4f40922 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:21.467282 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62196775-7a4c-42eb-8b44-3847d4f40922 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:21.467924 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62196775-7a4c-42eb-8b44-3847d4f40922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:21.533943 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-62196775-7a4c-42eb-8b44-3847d4f40922 None None] MFA Rules not processed for user `3f6d956add8045889ca198926b4eb7af`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:21.536511 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-62196775-7a4c-42eb-8b44-3847d4f40922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:21.554223 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:21.599128 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1321/2645] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:42:21.684592 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60138248-7200-45dd-9917-2d2cabc03e7c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:21.687778 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1325/2646] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:21.691835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62196775-7a4c-42eb-8b44-3847d4f40922 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:21.693383 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:21.694152 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-62196775-7a4c-42eb-8b44-3847d4f40922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:21.823809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62196775-7a4c-42eb-8b44-3847d4f40922 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:21.835519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4473e65-233e-4730-8137-05f5f7a3b802 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:21.836664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4473e65-233e-4730-8137-05f5f7a3b802 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:21.837159 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4473e65-233e-4730-8137-05f5f7a3b802 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:21.837722 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4473e65-233e-4730-8137-05f5f7a3b802 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:21.839361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60138248-7200-45dd-9917-2d2cabc03e7c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:21.848543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39928e5b-9a5f-4709-bde4-cc2e99724949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:21.851272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39928e5b-9a5f-4709-bde4-cc2e99724949 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:21.851272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39928e5b-9a5f-4709-bde4-cc2e99724949 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:21.851272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39928e5b-9a5f-4709-bde4-cc2e99724949 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:21.854361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39928e5b-9a5f-4709-bde4-cc2e99724949 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:21.857400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4473e65-233e-4730-8137-05f5f7a3b802 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:21.858376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39928e5b-9a5f-4709-bde4-cc2e99724949 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:21.860425 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4473e65-233e-4730-8137-05f5f7a3b802 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:21.888442 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1326/2647] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:42:21.989521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1322/2648] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:21 2022] POST /identity/v3/projects => generated 385 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:21.997668 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4473e65-233e-4730-8137-05f5f7a3b802 service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:22.016022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4473e65-233e-4730-8137-05f5f7a3b802 service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:22.108841 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39928e5b-9a5f-4709-bde4-cc2e99724949 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:22.111987 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-39928e5b-9a5f-4709-bde4-cc2e99724949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:22.138573 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6d794-e1ff-4f40-87ac-7519427690e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:22.139898 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6d794-e1ff-4f40-87ac-7519427690e7 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:22.140437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6d794-e1ff-4f40-87ac-7519427690e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:22.140840 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6d794-e1ff-4f40-87ac-7519427690e7 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:22.142043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6d794-e1ff-4f40-87ac-7519427690e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:22.145750 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6d794-e1ff-4f40-87ac-7519427690e7 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:22.217315 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39928e5b-9a5f-4709-bde4-cc2e99724949 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:22.223833 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0de8b-dab2-46fd-b510-5d949eed05a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:22.224907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0de8b-dab2-46fd-b510-5d949eed05a1 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:22.225407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0de8b-dab2-46fd-b510-5d949eed05a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:22.225982 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0de8b-dab2-46fd-b510-5d949eed05a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:22.343237 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1327/2649] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:21 2022] POST /identity/v3/users => generated 299 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:22.344949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0de8b-dab2-46fd-b510-5d949eed05a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:22.347889 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0de8b-dab2-46fd-b510-5d949eed05a1 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:22.349981 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6d794-e1ff-4f40-87ac-7519427690e7 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:22.366621 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1323/2650] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:42:22.369015 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6d794-e1ff-4f40-87ac-7519427690e7 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:22.378695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94def5d-6b68-40b3-a6ab-68dd4cd1dae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:22.379810 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94def5d-6b68-40b3-a6ab-68dd4cd1dae2 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:22.380322 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94def5d-6b68-40b3-a6ab-68dd4cd1dae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:22.380780 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94def5d-6b68-40b3-a6ab-68dd4cd1dae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:22.382143 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94def5d-6b68-40b3-a6ab-68dd4cd1dae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:22.385199 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94def5d-6b68-40b3-a6ab-68dd4cd1dae2 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:22.406392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1328/2651] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:22.413529 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0de8b-dab2-46fd-b510-5d949eed05a1 service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:22.435655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0de8b-dab2-46fd-b510-5d949eed05a1 service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:22.444561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538a2e9-c92e-4111-8d6f-f3315ad8e016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:22.445766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538a2e9-c92e-4111-8d6f-f3315ad8e016 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:22.446373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538a2e9-c92e-4111-8d6f-f3315ad8e016 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:22.446871 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538a2e9-c92e-4111-8d6f-f3315ad8e016 admin admin] PATH_INFO: `/v3/projects/6f5dbf4c6ae1426fa94302b26e23062b/users/634d91a206a947b691f42befd1737b2b/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:22.479831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538a2e9-c92e-4111-8d6f-f3315ad8e016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5dbf4c6ae1426fa94302b26e23062b, user_id=634d91a206a947b691f42befd1737b2b, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:22.486175 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538a2e9-c92e-4111-8d6f-f3315ad8e016 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:22.713819 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1324/2652] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:22 2022] PUT /identity/v3/projects/6f5dbf4c6ae1426fa94302b26e23062b/users/634d91a206a947b691f42befd1737b2b/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:22.718606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94def5d-6b68-40b3-a6ab-68dd4cd1dae2 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:22.732423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94def5d-6b68-40b3-a6ab-68dd4cd1dae2 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:22.783218 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:22.836510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff774014-4b79-4b83-a530-84006cc9ef30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:22.837563 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff774014-4b79-4b83-a530-84006cc9ef30 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:22.837563 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff774014-4b79-4b83-a530-84006cc9ef30 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:22.838015 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff774014-4b79-4b83-a530-84006cc9ef30 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:22.839280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff774014-4b79-4b83-a530-84006cc9ef30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:22.842535 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff774014-4b79-4b83-a530-84006cc9ef30 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:22.864927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1329/2653] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:22.872081 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538a2e9-c92e-4111-8d6f-f3315ad8e016 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:22.891603 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538a2e9-c92e-4111-8d6f-f3315ad8e016 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:22.906893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67261dd9-e5e3-4fb1-a5cb-0711cad1de20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:22.908488 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67261dd9-e5e3-4fb1-a5cb-0711cad1de20 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:22.909071 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67261dd9-e5e3-4fb1-a5cb-0711cad1de20 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:22.909653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67261dd9-e5e3-4fb1-a5cb-0711cad1de20 admin admin] PATH_INFO: `/v3/projects/6f5dbf4c6ae1426fa94302b26e23062b/users/634d91a206a947b691f42befd1737b2b/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:22.913514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67261dd9-e5e3-4fb1-a5cb-0711cad1de20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5dbf4c6ae1426fa94302b26e23062b, user_id=634d91a206a947b691f42befd1737b2b, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:22.919143 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67261dd9-e5e3-4fb1-a5cb-0711cad1de20 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:22.940353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1325/2654] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:22 2022] PUT /identity/v3/projects/6f5dbf4c6ae1426fa94302b26e23062b/users/634d91a206a947b691f42befd1737b2b/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:22.961020 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be66626-0279-4dad-982e-188546bd6f90 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:22.962038 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be66626-0279-4dad-982e-188546bd6f90 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:22.962509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be66626-0279-4dad-982e-188546bd6f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:23.018275 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-4be66626-0279-4dad-982e-188546bd6f90 None None] MFA Rules not processed for user `634d91a206a947b691f42befd1737b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:23.020638 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-4be66626-0279-4dad-982e-188546bd6f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:23.161871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1330/2655] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:23.769792 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67261dd9-e5e3-4fb1-a5cb-0711cad1de20 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:23.776964 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:23.868894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67261dd9-e5e3-4fb1-a5cb-0711cad1de20 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:23.876743 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f9c12-42fd-45ad-89b2-823c001f94c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:23.877455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f9c12-42fd-45ad-89b2-823c001f94c1 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:23.877934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f9c12-42fd-45ad-89b2-823c001f94c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:23.877934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f9c12-42fd-45ad-89b2-823c001f94c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:23.980268 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f9c12-42fd-45ad-89b2-823c001f94c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:23.981908 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f9c12-42fd-45ad-89b2-823c001f94c1 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:23.992343 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1326/2656] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:42:28.228410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be66626-0279-4dad-982e-188546bd6f90 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:28.230342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-4be66626-0279-4dad-982e-188546bd6f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:28.357297 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be66626-0279-4dad-982e-188546bd6f90 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:28.362335 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be6bc4-3032-46f3-874c-bf2db54e9ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:28.363019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be6bc4-3032-46f3-874c-bf2db54e9ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:28.363307 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be6bc4-3032-46f3-874c-bf2db54e9ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:28.363573 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be6bc4-3032-46f3-874c-bf2db54e9ee8 admin admin] PATH_INFO: `/v3/users/cab8ca33d0f040358d15eb24c8ee5c4b` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:28.365057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be6bc4-3032-46f3-874c-bf2db54e9ee8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cab8ca33d0f040358d15eb24c8ee5c4b)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:28.367059 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be6bc4-3032-46f3-874c-bf2db54e9ee8 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:28.621969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a0be6bc4-3032-46f3-874c-bf2db54e9ee8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cab8ca33d0f040358d15eb24c8ee5c4b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:28.722153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a0be6bc4-3032-46f3-874c-bf2db54e9ee8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cab8ca33d0f040358d15eb24c8ee5c4b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:28.724486 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a0be6bc4-3032-46f3-874c-bf2db54e9ee8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cab8ca33d0f040358d15eb24c8ee5c4b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:28.725889 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a0be6bc4-3032-46f3-874c-bf2db54e9ee8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cab8ca33d0f040358d15eb24c8ee5c4b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:28.731183 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:28.740776 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-a0be6bc4-3032-46f3-874c-bf2db54e9ee8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cab8ca33d0f040358d15eb24c8ee5c4b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:28.768479 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1331/2657] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:28 2022] DELETE /identity/v3/users/cab8ca33d0f040358d15eb24c8ee5c4b => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:28.791659 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca3597-2b08-4290-a4c4-36993d4dc4a4 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:28.793136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca3597-2b08-4290-a4c4-36993d4dc4a4 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:28.793136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca3597-2b08-4290-a4c4-36993d4dc4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:28.841780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-20ca3597-2b08-4290-a4c4-36993d4dc4a4 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:28.844959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-20ca3597-2b08-4290-a4c4-36993d4dc4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:28.965917 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1327/2658] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:28.981871 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be6bc4-3032-46f3-874c-bf2db54e9ee8 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:28.984550 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a0be6bc4-3032-46f3-874c-bf2db54e9ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:29.071966 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be6bc4-3032-46f3-874c-bf2db54e9ee8 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:29.077600 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0eeed-b59d-486d-9fdc-c95f0caba501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:29.078299 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0eeed-b59d-486d-9fdc-c95f0caba501 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:29.078618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0eeed-b59d-486d-9fdc-c95f0caba501 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:29.078948 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0eeed-b59d-486d-9fdc-c95f0caba501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:29.091698 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0eeed-b59d-486d-9fdc-c95f0caba501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:29.093349 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0eeed-b59d-486d-9fdc-c95f0caba501 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:29.104796 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1332/2659] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:42:29.701771 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca3597-2b08-4290-a4c4-36993d4dc4a4 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:29.704037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-20ca3597-2b08-4290-a4c4-36993d4dc4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:29.734767 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca3597-2b08-4290-a4c4-36993d4dc4a4 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:29.744132 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbef3bf-f953-453f-861b-b73d5792c65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:29.745294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbef3bf-f953-453f-861b-b73d5792c65c admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:29.745831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbef3bf-f953-453f-861b-b73d5792c65c admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:29.746285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbef3bf-f953-453f-861b-b73d5792c65c admin admin] PATH_INFO: `/v3/projects/e03f4c40f3d94baba4adc3e9e250b4e3` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:29.748373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbef3bf-f953-453f-861b-b73d5792c65c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e03f4c40f3d94baba4adc3e9e250b4e3)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:29.752244 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbef3bf-f953-453f-861b-b73d5792c65c admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:29.764671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0eeed-b59d-486d-9fdc-c95f0caba501 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:29.766851 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-a1a0eeed-b59d-486d-9fdc-c95f0caba501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:29.785341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0eeed-b59d-486d-9fdc-c95f0caba501 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:29.795436 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656cab2-0ef9-443e-a22d-f55ef399904d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:29.796855 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656cab2-0ef9-443e-a22d-f55ef399904d admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:29.797290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656cab2-0ef9-443e-a22d-f55ef399904d admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:29.797938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656cab2-0ef9-443e-a22d-f55ef399904d admin admin] PATH_INFO: `/v3/users/c9bb9aecd24c4a798e8f48a3e93040cb` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:29.799703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656cab2-0ef9-443e-a22d-f55ef399904d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9bb9aecd24c4a798e8f48a3e93040cb)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:29.801643 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656cab2-0ef9-443e-a22d-f55ef399904d admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:30.011703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-8dbef3bf-f953-453f-861b-b73d5792c65c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e03f4c40f3d94baba4adc3e9e250b4e3', 'request_id': 'req-8dbef3bf-f953-453f-861b-b73d5792c65c'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:30.032949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-8dbef3bf-f953-453f-861b-b73d5792c65c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e03f4c40f3d94baba4adc3e9e250b4e3', 'request_id': 'req-8dbef3bf-f953-453f-861b-b73d5792c65c'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:30.033303 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:30.103250 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0656cab2-0ef9-443e-a22d-f55ef399904d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9bb9aecd24c4a798e8f48a3e93040cb'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:30.217350 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-8dbef3bf-f953-453f-861b-b73d5792c65c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e03f4c40f3d94baba4adc3e9e250b4e3', 'request_id': 'req-8dbef3bf-f953-453f-861b-b73d5792c65c'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:30.236490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-8dbef3bf-f953-453f-861b-b73d5792c65c admin admin] The token cache is being invalidate because project e03f4c40f3d94baba4adc3e9e250b4e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:30.236968 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-8dbef3bf-f953-453f-861b-b73d5792c65c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:30.238767 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-8dbef3bf-f953-453f-861b-b73d5792c65c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:30.241541 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1328/2660] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:29 2022] DELETE /identity/v3/projects/e03f4c40f3d94baba4adc3e9e250b4e3 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:30.249085 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d7855-cc81-44cf-85cd-2f8a11f1d054 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:30.249563 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d7855-cc81-44cf-85cd-2f8a11f1d054 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:30.250040 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d7855-cc81-44cf-85cd-2f8a11f1d054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:30.282583 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-f87d7855-cc81-44cf-85cd-2f8a11f1d054 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:30.284341 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f87d7855-cc81-44cf-85cd-2f8a11f1d054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:30.319336 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0656cab2-0ef9-443e-a22d-f55ef399904d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9bb9aecd24c4a798e8f48a3e93040cb'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:30.319965 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0656cab2-0ef9-443e-a22d-f55ef399904d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9bb9aecd24c4a798e8f48a3e93040cb'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:30.321159 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0656cab2-0ef9-443e-a22d-f55ef399904d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9bb9aecd24c4a798e8f48a3e93040cb'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:30.329855 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0656cab2-0ef9-443e-a22d-f55ef399904d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9bb9aecd24c4a798e8f48a3e93040cb'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:30.369733 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1333/2661] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:29 2022] DELETE /identity/v3/users/c9bb9aecd24c4a798e8f48a3e93040cb => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:30.375371 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca3ef7-320a-429e-bc0c-f695f40271c6 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:30.375686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca3ef7-320a-429e-bc0c-f695f40271c6 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:30.376071 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca3ef7-320a-429e-bc0c-f695f40271c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:30.397696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1329/2662] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:42:30.405214 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-dcca3ef7-320a-429e-bc0c-f695f40271c6 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:30.406058 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-dcca3ef7-320a-429e-bc0c-f695f40271c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:30.414611 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53545d18-064f-4e18-99b8-39fc77e37dea None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:30.415167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53545d18-064f-4e18-99b8-39fc77e37dea None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:30.415510 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53545d18-064f-4e18-99b8-39fc77e37dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:30.448299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-53545d18-064f-4e18-99b8-39fc77e37dea None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:30.462901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-53545d18-064f-4e18-99b8-39fc77e37dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:30.485732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:30.506429 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1334/2663] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:42:30.522550 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca3ef7-320a-429e-bc0c-f695f40271c6 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:30.532064 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-dcca3ef7-320a-429e-bc0c-f695f40271c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:30.569313 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1330/2664] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:30.576272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53545d18-064f-4e18-99b8-39fc77e37dea None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:30.598127 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53545d18-064f-4e18-99b8-39fc77e37dea None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:30.611174 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7eb64-bb48-4dc7-b8f8-268d8df8ec85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:30.611888 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7eb64-bb48-4dc7-b8f8-268d8df8ec85 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:30.612339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7eb64-bb48-4dc7-b8f8-268d8df8ec85 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:30.612686 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7eb64-bb48-4dc7-b8f8-268d8df8ec85 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:30.613467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7eb64-bb48-4dc7-b8f8-268d8df8ec85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:42:30.613842 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7eb64-bb48-4dc7-b8f8-268d8df8ec85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:30.616482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7eb64-bb48-4dc7-b8f8-268d8df8ec85 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:30.653617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca3ef7-320a-429e-bc0c-f695f40271c6 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:30.661061 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1331/2665] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:42:30.667511 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7eb64-bb48-4dc7-b8f8-268d8df8ec85 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:30.681597 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7eb64-bb48-4dc7-b8f8-268d8df8ec85 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:30.689931 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76808b46-3055-48b6-98bb-6098fcf819bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:30.691266 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76808b46-3055-48b6-98bb-6098fcf819bd service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:30.691770 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76808b46-3055-48b6-98bb-6098fcf819bd service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:30.692456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76808b46-3055-48b6-98bb-6098fcf819bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:30.697364 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab664b31-823a-4c65-997e-9250c24829a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:30.700734 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab664b31-823a-4c65-997e-9250c24829a7 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:30.700734 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab664b31-823a-4c65-997e-9250c24829a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:30.700734 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab664b31-823a-4c65-997e-9250c24829a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:30.700734 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab664b31-823a-4c65-997e-9250c24829a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:30.701897 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab664b31-823a-4c65-997e-9250c24829a7 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:30.710085 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76808b46-3055-48b6-98bb-6098fcf819bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:30.712217 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76808b46-3055-48b6-98bb-6098fcf819bd service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:30.734716 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1335/2666] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:42:30.833831 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1332/2667] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:30 2022] POST /identity/v3/projects => generated 397 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:42:30.838879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76808b46-3055-48b6-98bb-6098fcf819bd service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:30.857588 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76808b46-3055-48b6-98bb-6098fcf819bd service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:30.941077 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ea977-3d85-41a0-aa94-c6421e53ca40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:30.941707 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ea977-3d85-41a0-aa94-c6421e53ca40 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:30.941986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ea977-3d85-41a0-aa94-c6421e53ca40 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:30.942254 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ea977-3d85-41a0-aa94-c6421e53ca40 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:30.942902 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ea977-3d85-41a0-aa94-c6421e53ca40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:30.944916 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ea977-3d85-41a0-aa94-c6421e53ca40 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:31.102660 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1336/2668] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:30 2022] POST /identity/v3/users => generated 299 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:31.108932 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab664b31-823a-4c65-997e-9250c24829a7 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:31.125978 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab664b31-823a-4c65-997e-9250c24829a7 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:31.132041 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718d410-7258-4114-9747-65c9c89f91f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:31.132754 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718d410-7258-4114-9747-65c9c89f91f7 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:31.133234 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718d410-7258-4114-9747-65c9c89f91f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:31.133571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718d410-7258-4114-9747-65c9c89f91f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:31.134299 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718d410-7258-4114-9747-65c9c89f91f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:31.137367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718d410-7258-4114-9747-65c9c89f91f7 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:31.151726 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1333/2669] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:31.157043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ea977-3d85-41a0-aa94-c6421e53ca40 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:31.169273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ea977-3d85-41a0-aa94-c6421e53ca40 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:31.175553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136d0cf-cfa7-4e3a-afdb-8c8ffa85bcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:31.176720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136d0cf-cfa7-4e3a-afdb-8c8ffa85bcde admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:31.177099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136d0cf-cfa7-4e3a-afdb-8c8ffa85bcde admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:31.177483 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136d0cf-cfa7-4e3a-afdb-8c8ffa85bcde admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:31.178265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136d0cf-cfa7-4e3a-afdb-8c8ffa85bcde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:31.180770 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136d0cf-cfa7-4e3a-afdb-8c8ffa85bcde admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:31.193259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1337/2670] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:31.198890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718d410-7258-4114-9747-65c9c89f91f7 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:31.210873 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718d410-7258-4114-9747-65c9c89f91f7 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:31.216257 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be23ede-6a70-4b2d-a03e-7fce8e8f24b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:31.216959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be23ede-6a70-4b2d-a03e-7fce8e8f24b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:31.217238 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be23ede-6a70-4b2d-a03e-7fce8e8f24b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:31.217657 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be23ede-6a70-4b2d-a03e-7fce8e8f24b0 admin admin] PATH_INFO: `/v3/projects/4eef4acdfbfe417493bcedcdc78b0f69/users/d8cd82c577384a4b881bea09cb317535/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:31.246645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be23ede-6a70-4b2d-a03e-7fce8e8f24b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eef4acdfbfe417493bcedcdc78b0f69, user_id=d8cd82c577384a4b881bea09cb317535, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:31.251380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be23ede-6a70-4b2d-a03e-7fce8e8f24b0 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:31.299967 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136d0cf-cfa7-4e3a-afdb-8c8ffa85bcde admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:31.302037 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8136d0cf-cfa7-4e3a-afdb-8c8ffa85bcde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:31.338147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136d0cf-cfa7-4e3a-afdb-8c8ffa85bcde admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:31.338147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90104274-0a4a-4f28-84ad-fe723b074961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:31.338147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90104274-0a4a-4f28-84ad-fe723b074961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:31.338147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90104274-0a4a-4f28-84ad-fe723b074961 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:31.339412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90104274-0a4a-4f28-84ad-fe723b074961 admin admin] PATH_INFO: `/v3/projects/bf2003df716e424f888324b8227da9f8` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:31.339412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90104274-0a4a-4f28-84ad-fe723b074961 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf2003df716e424f888324b8227da9f8)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:31.341916 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90104274-0a4a-4f28-84ad-fe723b074961 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:31.424319 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1334/2671] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:31 2022] PUT /identity/v3/projects/4eef4acdfbfe417493bcedcdc78b0f69/users/d8cd82c577384a4b881bea09cb317535/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:31.446691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f73d9b-9dfa-481c-992a-d65e25be7b7d None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:31.446691 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f73d9b-9dfa-481c-992a-d65e25be7b7d None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:31.452487 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f73d9b-9dfa-481c-992a-d65e25be7b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:31.494696 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:31.501526 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-19f73d9b-9dfa-481c-992a-d65e25be7b7d None None] MFA Rules not processed for user `d8cd82c577384a4b881bea09cb317535`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:31.503890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-19f73d9b-9dfa-481c-992a-d65e25be7b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:31.531219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-90104274-0a4a-4f28-84ad-fe723b074961 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf2003df716e424f888324b8227da9f8', 'request_id': 'req-90104274-0a4a-4f28-84ad-fe723b074961'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:31.540689 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-90104274-0a4a-4f28-84ad-fe723b074961 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf2003df716e424f888324b8227da9f8', 'request_id': 'req-90104274-0a4a-4f28-84ad-fe723b074961'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:31.614397 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1335/2672] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:31 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:31.631258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f73d9b-9dfa-481c-992a-d65e25be7b7d None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:31.633380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-19f73d9b-9dfa-481c-992a-d65e25be7b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:31.646259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-90104274-0a4a-4f28-84ad-fe723b074961 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf2003df716e424f888324b8227da9f8', 'request_id': 'req-90104274-0a4a-4f28-84ad-fe723b074961'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:31.650270 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:31.662629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-90104274-0a4a-4f28-84ad-fe723b074961 admin admin] The token cache is being invalidate because project bf2003df716e424f888324b8227da9f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:31.662910 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-90104274-0a4a-4f28-84ad-fe723b074961 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:31.663687 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-90104274-0a4a-4f28-84ad-fe723b074961 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:31.666909 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1338/2673] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:31 2022] DELETE /identity/v3/projects/bf2003df716e424f888324b8227da9f8 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:31.671915 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90104274-0a4a-4f28-84ad-fe723b074961 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:31.672964 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-90104274-0a4a-4f28-84ad-fe723b074961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:31.742479 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f73d9b-9dfa-481c-992a-d65e25be7b7d None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:31.823075 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90104274-0a4a-4f28-84ad-fe723b074961 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:31.823075 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819bdb9a-9b42-46a0-91e6-30203a8dabc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:31.823075 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819bdb9a-9b42-46a0-91e6-30203a8dabc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:31.823075 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819bdb9a-9b42-46a0-91e6-30203a8dabc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:31.823717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819bdb9a-9b42-46a0-91e6-30203a8dabc2 admin admin] PATH_INFO: `/v3/users/55801f07dc1f45d684bf46a19cdb1fad` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:31.823717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819bdb9a-9b42-46a0-91e6-30203a8dabc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55801f07dc1f45d684bf46a19cdb1fad)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:31.823717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819bdb9a-9b42-46a0-91e6-30203a8dabc2 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:31.830398 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daceed1-26bf-4d6e-89be-269a66ac4d30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:31.831733 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daceed1-26bf-4d6e-89be-269a66ac4d30 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:31.832263 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daceed1-26bf-4d6e-89be-269a66ac4d30 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:31.832751 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daceed1-26bf-4d6e-89be-269a66ac4d30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:31.838267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-4daceed1-26bf-4d6e-89be-269a66ac4d30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:31.918011 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daceed1-26bf-4d6e-89be-269a66ac4d30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:31.919666 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daceed1-26bf-4d6e-89be-269a66ac4d30 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:31.930684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1336/2674] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:31 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:42:32.159984 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-819bdb9a-9b42-46a0-91e6-30203a8dabc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55801f07dc1f45d684bf46a19cdb1fad'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:32.307948 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-819bdb9a-9b42-46a0-91e6-30203a8dabc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55801f07dc1f45d684bf46a19cdb1fad'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:32.308582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-819bdb9a-9b42-46a0-91e6-30203a8dabc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55801f07dc1f45d684bf46a19cdb1fad'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:32.309204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-819bdb9a-9b42-46a0-91e6-30203a8dabc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55801f07dc1f45d684bf46a19cdb1fad'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:32.317063 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-819bdb9a-9b42-46a0-91e6-30203a8dabc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55801f07dc1f45d684bf46a19cdb1fad'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:32.338484 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1339/2675] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:31 2022] DELETE /identity/v3/users/55801f07dc1f45d684bf46a19cdb1fad => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:33.012904 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daceed1-26bf-4d6e-89be-269a66ac4d30 service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:33.050712 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daceed1-26bf-4d6e-89be-269a66ac4d30 service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:33.063757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819bdb9a-9b42-46a0-91e6-30203a8dabc2 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:33.065358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-819bdb9a-9b42-46a0-91e6-30203a8dabc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:33.076658 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:33.191384 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea637936-7315-4e19-9702-80f2b70902c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:33.191384 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea637936-7315-4e19-9702-80f2b70902c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:33.191384 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea637936-7315-4e19-9702-80f2b70902c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:33.191384 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea637936-7315-4e19-9702-80f2b70902c8 admin admin] PATH_INFO: `/v3/projects/255a71c8d92448f2910bbfbb026df440` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:33.192213 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea637936-7315-4e19-9702-80f2b70902c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=255a71c8d92448f2910bbfbb026df440)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:33.194359 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea637936-7315-4e19-9702-80f2b70902c8 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:33.203408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819bdb9a-9b42-46a0-91e6-30203a8dabc2 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:33.213018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947e471-d904-406f-b9c9-b002f39b5ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:33.214570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947e471-d904-406f-b9c9-b002f39b5ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:33.214570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947e471-d904-406f-b9c9-b002f39b5ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:33.217350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947e471-d904-406f-b9c9-b002f39b5ac1 admin admin] PATH_INFO: `/v3/users/2256b87107284890a7c1ac15fd26a0aa` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:33.217790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947e471-d904-406f-b9c9-b002f39b5ac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2256b87107284890a7c1ac15fd26a0aa)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:33.221685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947e471-d904-406f-b9c9-b002f39b5ac1 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:33.248575 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:33.488909 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-6947e471-d904-406f-b9c9-b002f39b5ac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2256b87107284890a7c1ac15fd26a0aa'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:33.490007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ea637936-7315-4e19-9702-80f2b70902c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '255a71c8d92448f2910bbfbb026df440', 'request_id': 'req-ea637936-7315-4e19-9702-80f2b70902c8'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:33.499514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ea637936-7315-4e19-9702-80f2b70902c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '255a71c8d92448f2910bbfbb026df440', 'request_id': 'req-ea637936-7315-4e19-9702-80f2b70902c8'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:33.582776 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-6947e471-d904-406f-b9c9-b002f39b5ac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2256b87107284890a7c1ac15fd26a0aa'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:33.583562 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-6947e471-d904-406f-b9c9-b002f39b5ac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2256b87107284890a7c1ac15fd26a0aa'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:33.584276 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-6947e471-d904-406f-b9c9-b002f39b5ac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2256b87107284890a7c1ac15fd26a0aa'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:33.593457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-6947e471-d904-406f-b9c9-b002f39b5ac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2256b87107284890a7c1ac15fd26a0aa'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:33.617838 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1340/2676] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:33 2022] DELETE /identity/v3/users/2256b87107284890a7c1ac15fd26a0aa => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:33.619114 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947e471-d904-406f-b9c9-b002f39b5ac1 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:33.620843 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-6947e471-d904-406f-b9c9-b002f39b5ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:33.658799 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ea637936-7315-4e19-9702-80f2b70902c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '255a71c8d92448f2910bbfbb026df440', 'request_id': 'req-ea637936-7315-4e19-9702-80f2b70902c8'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:33.676528 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ea637936-7315-4e19-9702-80f2b70902c8 admin admin] The token cache is being invalidate because project 255a71c8d92448f2910bbfbb026df440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:33.677230 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ea637936-7315-4e19-9702-80f2b70902c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:33.678715 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-ea637936-7315-4e19-9702-80f2b70902c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:33.685803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1337/2677] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:33 2022] DELETE /identity/v3/projects/255a71c8d92448f2910bbfbb026df440 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:33.685803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc511a10-8efc-4e0d-9ba6-18b02c982388 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:33.685803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc511a10-8efc-4e0d-9ba6-18b02c982388 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:33.685803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc511a10-8efc-4e0d-9ba6-18b02c982388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:33.712450 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947e471-d904-406f-b9c9-b002f39b5ac1 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:33.718574 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-dc511a10-8efc-4e0d-9ba6-18b02c982388 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:33.718923 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd88c8a-439a-462d-8126-461a7683b4ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:33.719273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd88c8a-439a-462d-8126-461a7683b4ca service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:33.719552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd88c8a-439a-462d-8126-461a7683b4ca service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:33.719820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd88c8a-439a-462d-8126-461a7683b4ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:33.720096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-dc511a10-8efc-4e0d-9ba6-18b02c982388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:33.722132 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ebd88c8a-439a-462d-8126-461a7683b4ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:33.831681 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd88c8a-439a-462d-8126-461a7683b4ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:33.833553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd88c8a-439a-462d-8126-461a7683b4ca service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:33.835605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1338/2678] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:33.837629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aeed58-7a27-4edd-8be7-ce0e25b66bd1 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:33.838098 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aeed58-7a27-4edd-8be7-ce0e25b66bd1 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:33.838529 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aeed58-7a27-4edd-8be7-ce0e25b66bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:33.855548 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1341/2679] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:42:33.857714 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd88c8a-439a-462d-8126-461a7683b4ca service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:33.859854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ebd88c8a-439a-462d-8126-461a7683b4ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:33.879500 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c2aeed58-7a27-4edd-8be7-ce0e25b66bd1 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:33.881251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c2aeed58-7a27-4edd-8be7-ce0e25b66bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:33.900101 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1339/2680] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:33.909421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d144e2-ceb6-44ab-93ff-23f6e1a5c1f5 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:33.910033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d144e2-ceb6-44ab-93ff-23f6e1a5c1f5 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:33.910615 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d144e2-ceb6-44ab-93ff-23f6e1a5c1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:33.951568 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-f6d144e2-ceb6-44ab-93ff-23f6e1a5c1f5 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:33.953352 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:33.954447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f6d144e2-ceb6-44ab-93ff-23f6e1a5c1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:33.971835 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1340/2681] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:33.974432 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd88c8a-439a-462d-8126-461a7683b4ca service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:33.976468 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d144e2-ceb6-44ab-93ff-23f6e1a5c1f5 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:33.979155 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb22444-2d01-4b88-bf45-2d954e517096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:33.979930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb22444-2d01-4b88-bf45-2d954e517096 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:33.980510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb22444-2d01-4b88-bf45-2d954e517096 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:33.980865 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb22444-2d01-4b88-bf45-2d954e517096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:33.992916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d144e2-ceb6-44ab-93ff-23f6e1a5c1f5 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:33.995926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb22444-2d01-4b88-bf45-2d954e517096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:33.999070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb22444-2d01-4b88-bf45-2d954e517096 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:34.003136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8d710-3035-4106-a66b-efce797c9d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:34.003136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8d710-3035-4106-a66b-efce797c9d90 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:34.003136 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8d710-3035-4106-a66b-efce797c9d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:34.003732 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8d710-3035-4106-a66b-efce797c9d90 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:34.004441 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8d710-3035-4106-a66b-efce797c9d90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:42:34.004930 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8d710-3035-4106-a66b-efce797c9d90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:34.009265 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8d710-3035-4106-a66b-efce797c9d90 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:34.019418 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1342/2682] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:42:34.035576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1341/2683] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:42:34.042505 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb22444-2d01-4b88-bf45-2d954e517096 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:34.057994 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb22444-2d01-4b88-bf45-2d954e517096 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:34.065211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacedb5-57ef-40e3-917f-df066bd377c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:34.067204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacedb5-57ef-40e3-917f-df066bd377c6 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:34.067916 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacedb5-57ef-40e3-917f-df066bd377c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:34.068751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacedb5-57ef-40e3-917f-df066bd377c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:34.071060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eacedb5-57ef-40e3-917f-df066bd377c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:34.074150 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eacedb5-57ef-40e3-917f-df066bd377c6 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:34.285987 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1343/2684] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:34 2022] POST /identity/v3/projects => generated 377 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:34.290936 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8d710-3035-4106-a66b-efce797c9d90 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:34.302311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8d710-3035-4106-a66b-efce797c9d90 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:34.330343 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:34.419251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b4fc0-5a51-4b5b-b232-73111ec627f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:34.420303 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b4fc0-5a51-4b5b-b232-73111ec627f6 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:34.421406 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b4fc0-5a51-4b5b-b232-73111ec627f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:34.421921 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b4fc0-5a51-4b5b-b232-73111ec627f6 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:34.422934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b4fc0-5a51-4b5b-b232-73111ec627f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:34.425160 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b4fc0-5a51-4b5b-b232-73111ec627f6 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:34.551425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1342/2685] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:34 2022] POST /identity/v3/users => generated 289 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:42:34.558363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacedb5-57ef-40e3-917f-df066bd377c6 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:34.576522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacedb5-57ef-40e3-917f-df066bd377c6 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:34.588538 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9aeba-f582-4775-87f1-02d1670b7215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:34.589751 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9aeba-f582-4775-87f1-02d1670b7215 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:34.590124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9aeba-f582-4775-87f1-02d1670b7215 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:34.590588 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9aeba-f582-4775-87f1-02d1670b7215 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:34.591863 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9aeba-f582-4775-87f1-02d1670b7215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:34.594682 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9aeba-f582-4775-87f1-02d1670b7215 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:34.618811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1344/2686] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:34.624657 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b4fc0-5a51-4b5b-b232-73111ec627f6 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:34.636869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b4fc0-5a51-4b5b-b232-73111ec627f6 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:34.643167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec0c83-45c1-443d-8fc1-61fd1baf7862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:34.643833 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec0c83-45c1-443d-8fc1-61fd1baf7862 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:34.644118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec0c83-45c1-443d-8fc1-61fd1baf7862 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:34.649111 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec0c83-45c1-443d-8fc1-61fd1baf7862 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:34.649111 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec0c83-45c1-443d-8fc1-61fd1baf7862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:34.649111 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec0c83-45c1-443d-8fc1-61fd1baf7862 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:34.659198 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1343/2687] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:34.665026 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9aeba-f582-4775-87f1-02d1670b7215 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:34.689819 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9aeba-f582-4775-87f1-02d1670b7215 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:34.698891 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6aee18-a120-4bde-a59d-167cb04d5630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:34.699835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6aee18-a120-4bde-a59d-167cb04d5630 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:34.700285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6aee18-a120-4bde-a59d-167cb04d5630 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:34.700836 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6aee18-a120-4bde-a59d-167cb04d5630 admin admin] PATH_INFO: `/v3/projects/4660fe0f55ef4c7cba4ee7374d2422ee/users/9aec64f63f5b4e09bcd02ab62444057d/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:34.728603 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec0c83-45c1-443d-8fc1-61fd1baf7862 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:34.732607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d8ec0c83-45c1-443d-8fc1-61fd1baf7862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:34.741900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6aee18-a120-4bde-a59d-167cb04d5630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4660fe0f55ef4c7cba4ee7374d2422ee, user_id=9aec64f63f5b4e09bcd02ab62444057d, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:34.754224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6aee18-a120-4bde-a59d-167cb04d5630 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:34.760203 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec0c83-45c1-443d-8fc1-61fd1baf7862 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:34.768588 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc416c0-1734-4c3a-8c2e-da344757e70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:34.769719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc416c0-1734-4c3a-8c2e-da344757e70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:34.770342 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc416c0-1734-4c3a-8c2e-da344757e70b admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:34.770833 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc416c0-1734-4c3a-8c2e-da344757e70b admin admin] PATH_INFO: `/v3/projects/652162682a1a4ab3b8dc548401f7dfe3` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:34.773408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc416c0-1734-4c3a-8c2e-da344757e70b admin admin] RBAC: Authorizing `identity:delete_project(project_id=652162682a1a4ab3b8dc548401f7dfe3)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:34.777347 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc416c0-1734-4c3a-8c2e-da344757e70b admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:34.885453 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1345/2688] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:34 2022] PUT /identity/v3/projects/4660fe0f55ef4c7cba4ee7374d2422ee/users/9aec64f63f5b4e09bcd02ab62444057d/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:34.907783 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04067f-3471-489a-9821-8eb122b86c69 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:34.908286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04067f-3471-489a-9821-8eb122b86c69 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:34.908772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04067f-3471-489a-9821-8eb122b86c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:34.960115 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-6f04067f-3471-489a-9821-8eb122b86c69 None None] MFA Rules not processed for user `9aec64f63f5b4e09bcd02ab62444057d`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:34.962632 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-6f04067f-3471-489a-9821-8eb122b86c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:35.009480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2bc416c0-1734-4c3a-8c2e-da344757e70b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '652162682a1a4ab3b8dc548401f7dfe3', 'request_id': 'req-2bc416c0-1734-4c3a-8c2e-da344757e70b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:35.021779 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2bc416c0-1734-4c3a-8c2e-da344757e70b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '652162682a1a4ab3b8dc548401f7dfe3', 'request_id': 'req-2bc416c0-1734-4c3a-8c2e-da344757e70b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:35.084127 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1346/2689] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:42:35.089803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04067f-3471-489a-9821-8eb122b86c69 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:35.119814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2bc416c0-1734-4c3a-8c2e-da344757e70b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '652162682a1a4ab3b8dc548401f7dfe3', 'request_id': 'req-2bc416c0-1734-4c3a-8c2e-da344757e70b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:35.139027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2bc416c0-1734-4c3a-8c2e-da344757e70b admin admin] The token cache is being invalidate because project 652162682a1a4ab3b8dc548401f7dfe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:35.139600 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2bc416c0-1734-4c3a-8c2e-da344757e70b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:35.141024 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-2bc416c0-1734-4c3a-8c2e-da344757e70b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:35.143387 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1344/2690] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:34 2022] DELETE /identity/v3/projects/652162682a1a4ab3b8dc548401f7dfe3 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:35.148232 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc416c0-1734-4c3a-8c2e-da344757e70b admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:35.150367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2bc416c0-1734-4c3a-8c2e-da344757e70b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:35.188566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04067f-3471-489a-9821-8eb122b86c69 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:35.214198 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:35.268261 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc416c0-1734-4c3a-8c2e-da344757e70b admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:35.270407 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1aa4e8-30fb-4630-8d1d-cff481372176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:35.271067 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1aa4e8-30fb-4630-8d1d-cff481372176 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:35.271354 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1aa4e8-30fb-4630-8d1d-cff481372176 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:35.271748 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1aa4e8-30fb-4630-8d1d-cff481372176 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:35.272452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1aa4e8-30fb-4630-8d1d-cff481372176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:35.274700 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb73a6d-6b6f-4ec0-8f50-01b587cff04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:35.275420 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb73a6d-6b6f-4ec0-8f50-01b587cff04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:35.275693 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb73a6d-6b6f-4ec0-8f50-01b587cff04b admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:35.275959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb73a6d-6b6f-4ec0-8f50-01b587cff04b admin admin] PATH_INFO: `/v3/users/445808ae2bfd4c94a4d8a1b71a59a5af` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:35.277385 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb73a6d-6b6f-4ec0-8f50-01b587cff04b admin admin] RBAC: Authorizing `identity:delete_user(user_id=445808ae2bfd4c94a4d8a1b71a59a5af)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:35.279451 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb73a6d-6b6f-4ec0-8f50-01b587cff04b admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:35.281050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1aa4e8-30fb-4630-8d1d-cff481372176 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:35.308449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:35.431777 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1347/2691] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:35 2022] POST /identity/v3/projects => generated 375 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:42:35.436571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1aa4e8-30fb-4630-8d1d-cff481372176 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:35.439531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-6b1aa4e8-30fb-4630-8d1d-cff481372176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:35.541796 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1aa4e8-30fb-4630-8d1d-cff481372176 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:35.547466 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37993f70-7e73-4e4f-afb4-f5d0c5c0781e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:35.548074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37993f70-7e73-4e4f-afb4-f5d0c5c0781e admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:35.548493 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37993f70-7e73-4e4f-afb4-f5d0c5c0781e admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:35.548771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37993f70-7e73-4e4f-afb4-f5d0c5c0781e admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:35.549430 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37993f70-7e73-4e4f-afb4-f5d0c5c0781e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:35.551440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37993f70-7e73-4e4f-afb4-f5d0c5c0781e admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:35.616679 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-bbb73a6d-6b6f-4ec0-8f50-01b587cff04b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '445808ae2bfd4c94a4d8a1b71a59a5af'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:35.630358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-bbb73a6d-6b6f-4ec0-8f50-01b587cff04b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '445808ae2bfd4c94a4d8a1b71a59a5af'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:35.631224 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-bbb73a6d-6b6f-4ec0-8f50-01b587cff04b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '445808ae2bfd4c94a4d8a1b71a59a5af'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:35.640487 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-bbb73a6d-6b6f-4ec0-8f50-01b587cff04b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '445808ae2bfd4c94a4d8a1b71a59a5af'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:35.641258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-bbb73a6d-6b6f-4ec0-8f50-01b587cff04b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '445808ae2bfd4c94a4d8a1b71a59a5af'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:35.671955 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1348/2692] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:35 2022] POST /identity/v3/users => generated 288 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:35.676574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37993f70-7e73-4e4f-afb4-f5d0c5c0781e admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:35.679233 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-37993f70-7e73-4e4f-afb4-f5d0c5c0781e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:35.703576 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37993f70-7e73-4e4f-afb4-f5d0c5c0781e admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:35.713193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cac55f-b1d1-47d9-bc2b-4daa9ff03f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:35.714281 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cac55f-b1d1-47d9-bc2b-4daa9ff03f8a admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:35.714784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cac55f-b1d1-47d9-bc2b-4daa9ff03f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:35.715330 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cac55f-b1d1-47d9-bc2b-4daa9ff03f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:35.716596 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cac55f-b1d1-47d9-bc2b-4daa9ff03f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:35.719581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cac55f-b1d1-47d9-bc2b-4daa9ff03f8a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:35.744973 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1349/2693] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:42:35.746899 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cac55f-b1d1-47d9-bc2b-4daa9ff03f8a admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:35.764377 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cac55f-b1d1-47d9-bc2b-4daa9ff03f8a admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:35.769433 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1345/2694] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:35 2022] DELETE /identity/v3/users/445808ae2bfd4c94a4d8a1b71a59a5af => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:35.772148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7acdd-5da0-4d19-8455-12b09d9bf89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:35.773323 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7acdd-5da0-4d19-8455-12b09d9bf89d admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:35.774325 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7acdd-5da0-4d19-8455-12b09d9bf89d admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:35.774623 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7acdd-5da0-4d19-8455-12b09d9bf89d admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:35.775947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7acdd-5da0-4d19-8455-12b09d9bf89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:35.778123 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7acdd-5da0-4d19-8455-12b09d9bf89d admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:35.794553 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1350/2695] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:42:35.801091 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb73a6d-6b6f-4ec0-8f50-01b587cff04b admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:35.895800 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb73a6d-6b6f-4ec0-8f50-01b587cff04b admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:35.912176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6df8d5-73ea-4be4-b093-b8d7c0986b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:35.913270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6df8d5-73ea-4be4-b093-b8d7c0986b3f admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:35.913735 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6df8d5-73ea-4be4-b093-b8d7c0986b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:35.914543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6df8d5-73ea-4be4-b093-b8d7c0986b3f admin admin] PATH_INFO: `/v3/projects/cca5d10d42844de78f582e78089e15f4/users/cb55a5a35b8a479fbc1ba3d1d284ff1d/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:35.934508 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6df8d5-73ea-4be4-b093-b8d7c0986b3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca5d10d42844de78f582e78089e15f4, user_id=cb55a5a35b8a479fbc1ba3d1d284ff1d, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:35.937425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6df8d5-73ea-4be4-b093-b8d7c0986b3f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:36.075845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1346/2696] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:35 2022] PUT /identity/v3/projects/cca5d10d42844de78f582e78089e15f4/users/cb55a5a35b8a479fbc1ba3d1d284ff1d/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:36.088210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef61128-a8e6-4115-9ac1-e8a032e90c26 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:36.089574 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef61128-a8e6-4115-9ac1-e8a032e90c26 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:36.093223 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef61128-a8e6-4115-9ac1-e8a032e90c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:36.127495 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-9ef61128-a8e6-4115-9ac1-e8a032e90c26 None None] MFA Rules not processed for user `cb55a5a35b8a479fbc1ba3d1d284ff1d`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:36.129183 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9ef61128-a8e6-4115-9ac1-e8a032e90c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:36.236612 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1351/2697] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:36.244013 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6df8d5-73ea-4be4-b093-b8d7c0986b3f admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:36.369883 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6df8d5-73ea-4be4-b093-b8d7c0986b3f admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:36.377973 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb1a95-4aaa-4c55-a56c-73495ac713dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:36.378962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb1a95-4aaa-4c55-a56c-73495ac713dc admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:36.379467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb1a95-4aaa-4c55-a56c-73495ac713dc admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:36.380087 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb1a95-4aaa-4c55-a56c-73495ac713dc admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:36.381152 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb1a95-4aaa-4c55-a56c-73495ac713dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:36.384596 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb1a95-4aaa-4c55-a56c-73495ac713dc admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:36.427243 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef61128-a8e6-4115-9ac1-e8a032e90c26 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:36.429803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9ef61128-a8e6-4115-9ac1-e8a032e90c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:36.450673 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:36.459016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef61128-a8e6-4115-9ac1-e8a032e90c26 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:36.470076 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabf78e-b7c5-4c77-ba6d-ef178cd3b389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:36.470927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabf78e-b7c5-4c77-ba6d-ef178cd3b389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:36.471482 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabf78e-b7c5-4c77-ba6d-ef178cd3b389 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:36.479861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabf78e-b7c5-4c77-ba6d-ef178cd3b389 admin admin] PATH_INFO: `/v3/projects/54d2f0a81890433987e437afc8157cfb` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:36.479861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabf78e-b7c5-4c77-ba6d-ef178cd3b389 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54d2f0a81890433987e437afc8157cfb)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:36.479861 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cabf78e-b7c5-4c77-ba6d-ef178cd3b389 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:36.548835 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1347/2698] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:36 2022] POST /identity/v3/projects => generated 379 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:36.551264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb1a95-4aaa-4c55-a56c-73495ac713dc admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:36.562238 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:36.644438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb1a95-4aaa-4c55-a56c-73495ac713dc admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:36.650300 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db5b1d-177b-47e2-9c46-c8a3ef6405ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:36.650927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db5b1d-177b-47e2-9c46-c8a3ef6405ae admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:36.651241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db5b1d-177b-47e2-9c46-c8a3ef6405ae admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:36.651513 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db5b1d-177b-47e2-9c46-c8a3ef6405ae admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:36.652164 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db5b1d-177b-47e2-9c46-c8a3ef6405ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:36.654088 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db5b1d-177b-47e2-9c46-c8a3ef6405ae admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:36.694161 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5cabf78e-b7c5-4c77-ba6d-ef178cd3b389 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54d2f0a81890433987e437afc8157cfb', 'request_id': 'req-5cabf78e-b7c5-4c77-ba6d-ef178cd3b389'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:36.705018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5cabf78e-b7c5-4c77-ba6d-ef178cd3b389 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54d2f0a81890433987e437afc8157cfb', 'request_id': 'req-5cabf78e-b7c5-4c77-ba6d-ef178cd3b389'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:36.783249 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1348/2699] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:36 2022] POST /identity/v3/users => generated 296 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:36.787694 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db5b1d-177b-47e2-9c46-c8a3ef6405ae admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:36.799384 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db5b1d-177b-47e2-9c46-c8a3ef6405ae admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:36.811584 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5cabf78e-b7c5-4c77-ba6d-ef178cd3b389 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54d2f0a81890433987e437afc8157cfb', 'request_id': 'req-5cabf78e-b7c5-4c77-ba6d-ef178cd3b389'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:36.824353 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5cabf78e-b7c5-4c77-ba6d-ef178cd3b389 admin admin] The token cache is being invalidate because project 54d2f0a81890433987e437afc8157cfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:36.824759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5cabf78e-b7c5-4c77-ba6d-ef178cd3b389 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:36.826447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-5cabf78e-b7c5-4c77-ba6d-ef178cd3b389 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:36.828342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1352/2700] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:36 2022] DELETE /identity/v3/projects/54d2f0a81890433987e437afc8157cfb => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:36.834222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01bcab-aa9c-48b5-af1c-4a010bfa0c69 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:36.834222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01bcab-aa9c-48b5-af1c-4a010bfa0c69 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:36.834564 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01bcab-aa9c-48b5-af1c-4a010bfa0c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:36.857545 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-ee01bcab-aa9c-48b5-af1c-4a010bfa0c69 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:36.859355 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ee01bcab-aa9c-48b5-af1c-4a010bfa0c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:36.887672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f1f06-084a-42ec-92f4-77e55530b089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:36.888599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f1f06-084a-42ec-92f4-77e55530b089 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:36.888975 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f1f06-084a-42ec-92f4-77e55530b089 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:36.889417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f1f06-084a-42ec-92f4-77e55530b089 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:36.890188 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f1f06-084a-42ec-92f4-77e55530b089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:36.892848 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f1f06-084a-42ec-92f4-77e55530b089 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:36.906150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1349/2701] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:42:36.910822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f1f06-084a-42ec-92f4-77e55530b089 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:36.913330 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-289f1f06-084a-42ec-92f4-77e55530b089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:36.934234 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f1f06-084a-42ec-92f4-77e55530b089 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:36.942342 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6f29e-e067-40cd-834a-21c47f7d4e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:36.944637 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6f29e-e067-40cd-834a-21c47f7d4e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:36.944637 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6f29e-e067-40cd-834a-21c47f7d4e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:36.944637 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6f29e-e067-40cd-834a-21c47f7d4e74 admin admin] PATH_INFO: `/v3/projects/c60ff568e3094bb899b63b47dfa1bbd1/users/60ab27cde180418c9c35ab22e707d8eb/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:36.949789 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1353/2702] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:42:36.964531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904674b-00cc-4a82-acc6-e8b970a451ab None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:36.964869 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904674b-00cc-4a82-acc6-e8b970a451ab None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:36.965187 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904674b-00cc-4a82-acc6-e8b970a451ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:36.971429 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6f29e-e067-40cd-834a-21c47f7d4e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60ff568e3094bb899b63b47dfa1bbd1, user_id=60ab27cde180418c9c35ab22e707d8eb, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:36.974358 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6f29e-e067-40cd-834a-21c47f7d4e74 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:36.998103 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-7904674b-00cc-4a82-acc6-e8b970a451ab None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:36.999449 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7904674b-00cc-4a82-acc6-e8b970a451ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:37.013020 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1354/2703] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:37.015402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904674b-00cc-4a82-acc6-e8b970a451ab None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:37.027752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904674b-00cc-4a82-acc6-e8b970a451ab None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:37.035617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530af1dc-2f0f-40d1-b22b-77173247f219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:37.036433 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530af1dc-2f0f-40d1-b22b-77173247f219 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:37.036743 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530af1dc-2f0f-40d1-b22b-77173247f219 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:37.037193 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530af1dc-2f0f-40d1-b22b-77173247f219 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:37.038437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530af1dc-2f0f-40d1-b22b-77173247f219 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:42:37.039157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530af1dc-2f0f-40d1-b22b-77173247f219 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:37.042520 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530af1dc-2f0f-40d1-b22b-77173247f219 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:37.064905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1355/2704] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:42:37.072548 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530af1dc-2f0f-40d1-b22b-77173247f219 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:37.080561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1350/2705] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:36 2022] PUT /identity/v3/projects/c60ff568e3094bb899b63b47dfa1bbd1/users/60ab27cde180418c9c35ab22e707d8eb/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:37.094342 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6f29e-e067-40cd-834a-21c47f7d4e74 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:37.103360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530af1dc-2f0f-40d1-b22b-77173247f219 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:37.201972 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ffcf8-9838-44ea-8699-41e726241e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:37.203225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ffcf8-9838-44ea-8699-41e726241e33 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:37.203730 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ffcf8-9838-44ea-8699-41e726241e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:37.204247 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ffcf8-9838-44ea-8699-41e726241e33 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:37.205483 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ffcf8-9838-44ea-8699-41e726241e33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:37.209552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ffcf8-9838-44ea-8699-41e726241e33 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:37.224472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6f29e-e067-40cd-834a-21c47f7d4e74 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:37.232122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f58a84-48a2-4165-b11d-a663a60d6e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:37.232883 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f58a84-48a2-4165-b11d-a663a60d6e7c admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:37.233661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f58a84-48a2-4165-b11d-a663a60d6e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:37.233661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f58a84-48a2-4165-b11d-a663a60d6e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:37.234339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f58a84-48a2-4165-b11d-a663a60d6e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:37.236311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f58a84-48a2-4165-b11d-a663a60d6e7c admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:37.240619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:37.252813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1351/2706] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:42:37.259846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f58a84-48a2-4165-b11d-a663a60d6e7c admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:37.267547 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:37.296144 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f58a84-48a2-4165-b11d-a663a60d6e7c admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:37.296144 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437c4f2-caf3-41bb-97be-fd11de81f16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:37.296144 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437c4f2-caf3-41bb-97be-fd11de81f16d admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:37.296144 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437c4f2-caf3-41bb-97be-fd11de81f16d admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:37.296949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437c4f2-caf3-41bb-97be-fd11de81f16d admin admin] PATH_INFO: `/v3/projects/c60ff568e3094bb899b63b47dfa1bbd1/users/60ab27cde180418c9c35ab22e707d8eb/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:37.296949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8437c4f2-caf3-41bb-97be-fd11de81f16d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60ff568e3094bb899b63b47dfa1bbd1, user_id=60ab27cde180418c9c35ab22e707d8eb, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:37.296949 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8437c4f2-caf3-41bb-97be-fd11de81f16d admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:37.307236 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1352/2707] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:37 2022] PUT /identity/v3/projects/c60ff568e3094bb899b63b47dfa1bbd1/users/60ab27cde180418c9c35ab22e707d8eb/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:37.390271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ac31e-3688-4710-9f7a-f82884ed48f9 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:37.390554 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ac31e-3688-4710-9f7a-f82884ed48f9 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:37.390830 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ac31e-3688-4710-9f7a-f82884ed48f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:37.427278 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-dd6ac31e-3688-4710-9f7a-f82884ed48f9 None None] MFA Rules not processed for user `60ab27cde180418c9c35ab22e707d8eb`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:37.428668 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-dd6ac31e-3688-4710-9f7a-f82884ed48f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:37.513967 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1356/2708] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:37 2022] POST /identity/v3/projects => generated 387 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:37.520300 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ffcf8-9838-44ea-8699-41e726241e33 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:37.537331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1353/2709] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:42:37.552233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ac31e-3688-4710-9f7a-f82884ed48f9 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:37.556142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-dd6ac31e-3688-4710-9f7a-f82884ed48f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:37.625098 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ffcf8-9838-44ea-8699-41e726241e33 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:37.630541 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45239ea3-dbcd-4fa7-afc7-b7f3edcb4148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:37.631386 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45239ea3-dbcd-4fa7-afc7-b7f3edcb4148 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:37.631714 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45239ea3-dbcd-4fa7-afc7-b7f3edcb4148 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:37.640814 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45239ea3-dbcd-4fa7-afc7-b7f3edcb4148 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:37.640814 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45239ea3-dbcd-4fa7-afc7-b7f3edcb4148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:37.640814 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45239ea3-dbcd-4fa7-afc7-b7f3edcb4148 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:37.641426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ac31e-3688-4710-9f7a-f82884ed48f9 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:37.701720 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9352c-0c23-4a61-bceb-3b8006e74bce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:37.702268 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9352c-0c23-4a61-bceb-3b8006e74bce service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:37.702543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9352c-0c23-4a61-bceb-3b8006e74bce service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:37.702815 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9352c-0c23-4a61-bceb-3b8006e74bce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:37.753738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:37.777654 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9352c-0c23-4a61-bceb-3b8006e74bce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:37.780731 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9352c-0c23-4a61-bceb-3b8006e74bce service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:37.790582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1357/2710] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:37 2022] POST /identity/v3/users => generated 294 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:37.798319 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45239ea3-dbcd-4fa7-afc7-b7f3edcb4148 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:37.808142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1354/2711] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:37 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:42:37.820654 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45239ea3-dbcd-4fa7-afc7-b7f3edcb4148 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:37.827898 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06a4ec-3459-4f86-9ab9-989972e2da47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:37.828696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06a4ec-3459-4f86-9ab9-989972e2da47 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:37.829148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06a4ec-3459-4f86-9ab9-989972e2da47 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:37.829606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06a4ec-3459-4f86-9ab9-989972e2da47 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:37.830456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06a4ec-3459-4f86-9ab9-989972e2da47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:37.833350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06a4ec-3459-4f86-9ab9-989972e2da47 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:37.848956 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1358/2712] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:37.855550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9352c-0c23-4a61-bceb-3b8006e74bce service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:37.869001 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9352c-0c23-4a61-bceb-3b8006e74bce service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:37.876530 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54da3d1-1121-4e4e-a515-eef9d3e78d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:37.877173 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54da3d1-1121-4e4e-a515-eef9d3e78d90 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:37.877488 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54da3d1-1121-4e4e-a515-eef9d3e78d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:37.882556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54da3d1-1121-4e4e-a515-eef9d3e78d90 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:37.882556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54da3d1-1121-4e4e-a515-eef9d3e78d90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:37.882556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54da3d1-1121-4e4e-a515-eef9d3e78d90 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:37.893054 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1355/2713] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:37.898592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06a4ec-3459-4f86-9ab9-989972e2da47 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:37.912677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06a4ec-3459-4f86-9ab9-989972e2da47 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:37.918451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e345c8-b33f-4d0d-a992-f2788082ae7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:37.919179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e345c8-b33f-4d0d-a992-f2788082ae7c admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:37.919455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e345c8-b33f-4d0d-a992-f2788082ae7c admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:37.919794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e345c8-b33f-4d0d-a992-f2788082ae7c admin admin] PATH_INFO: `/v3/projects/1bea16d670ff4197a13f124a9a71b9cd/users/f6d5fb7b2db64b509043f9bb20d8f4ad/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:37.944768 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e345c8-b33f-4d0d-a992-f2788082ae7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bea16d670ff4197a13f124a9a71b9cd, user_id=f6d5fb7b2db64b509043f9bb20d8f4ad, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:37.950092 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e345c8-b33f-4d0d-a992-f2788082ae7c admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:38.041946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54da3d1-1121-4e4e-a515-eef9d3e78d90 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:38.044373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f54da3d1-1121-4e4e-a515-eef9d3e78d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:38.063514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54da3d1-1121-4e4e-a515-eef9d3e78d90 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:38.071139 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1359/2714] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:37 2022] PUT /identity/v3/projects/1bea16d670ff4197a13f124a9a71b9cd/users/f6d5fb7b2db64b509043f9bb20d8f4ad/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:38.083509 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5a3b8-6dca-4dc1-b39b-5c1e112024f7 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:38.084188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5a3b8-6dca-4dc1-b39b-5c1e112024f7 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:38.084303 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5a3b8-6dca-4dc1-b39b-5c1e112024f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:38.131877 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-9af5a3b8-6dca-4dc1-b39b-5c1e112024f7 None None] MFA Rules not processed for user `f6d5fb7b2db64b509043f9bb20d8f4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:38.134579 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9af5a3b8-6dca-4dc1-b39b-5c1e112024f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:38.147543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b7a20-7bbd-48c2-8d78-d4c54a064f96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:38.155704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b7a20-7bbd-48c2-8d78-d4c54a064f96 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:38.155704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b7a20-7bbd-48c2-8d78-d4c54a064f96 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:38.155704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b7a20-7bbd-48c2-8d78-d4c54a064f96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:38.155704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ab9b7a20-7bbd-48c2-8d78-d4c54a064f96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:38.255850 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b7a20-7bbd-48c2-8d78-d4c54a064f96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:38.255850 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b7a20-7bbd-48c2-8d78-d4c54a064f96 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:38.266318 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1360/2715] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:42:38.272886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1356/2716] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:42:38.274521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5a3b8-6dca-4dc1-b39b-5c1e112024f7 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:38.360298 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:38.395426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5a3b8-6dca-4dc1-b39b-5c1e112024f7 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:38.402397 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee746820-4c55-4720-92af-5fc019585079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:38.403261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee746820-4c55-4720-92af-5fc019585079 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:38.403712 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee746820-4c55-4720-92af-5fc019585079 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:38.404183 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee746820-4c55-4720-92af-5fc019585079 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:38.405391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee746820-4c55-4720-92af-5fc019585079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:38.409428 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee746820-4c55-4720-92af-5fc019585079 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:38.546309 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1361/2717] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:38 2022] POST /identity/v3/projects => generated 385 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:42:38.551397 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b7a20-7bbd-48c2-8d78-d4c54a064f96 service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:38.570661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b7a20-7bbd-48c2-8d78-d4c54a064f96 service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:38.676676 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8527a-c7cc-4ac9-991f-3455fff42772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:38.677458 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8527a-c7cc-4ac9-991f-3455fff42772 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:38.678089 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8527a-c7cc-4ac9-991f-3455fff42772 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:38.678384 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8527a-c7cc-4ac9-991f-3455fff42772 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:38.679254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8527a-c7cc-4ac9-991f-3455fff42772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:38.681661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8527a-c7cc-4ac9-991f-3455fff42772 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:38.890531 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1357/2718] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:38 2022] POST /identity/v3/users => generated 299 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:38.897559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee746820-4c55-4720-92af-5fc019585079 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:38.921366 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee746820-4c55-4720-92af-5fc019585079 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:38.928496 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7e907-1372-40d0-8c98-daf6818542f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:38.929387 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7e907-1372-40d0-8c98-daf6818542f8 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:38.930822 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7e907-1372-40d0-8c98-daf6818542f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:38.931255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7e907-1372-40d0-8c98-daf6818542f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:38.931881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7e907-1372-40d0-8c98-daf6818542f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:38.933737 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7e907-1372-40d0-8c98-daf6818542f8 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:38.946802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1362/2719] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:38.951882 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8527a-c7cc-4ac9-991f-3455fff42772 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:38.963930 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8527a-c7cc-4ac9-991f-3455fff42772 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:38.968724 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2aad5-221a-4884-a286-21a021d7e703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:38.969399 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2aad5-221a-4884-a286-21a021d7e703 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:38.969674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2aad5-221a-4884-a286-21a021d7e703 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:38.969982 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d2aad5-221a-4884-a286-21a021d7e703 admin admin] PATH_INFO: `/v3/projects/8118072c71ae4211aa74939e78fb4ef9/users/66a5da183c784821a2d924d9ab61b770/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:38.985290 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:38.989328 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2aad5-221a-4884-a286-21a021d7e703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8118072c71ae4211aa74939e78fb4ef9, user_id=66a5da183c784821a2d924d9ab61b770, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:38.993245 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d2aad5-221a-4884-a286-21a021d7e703 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:39.236092 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1358/2720] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:38 2022] PUT /identity/v3/projects/8118072c71ae4211aa74939e78fb4ef9/users/66a5da183c784821a2d924d9ab61b770/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:39.240467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7e907-1372-40d0-8c98-daf6818542f8 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:39.255364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7e907-1372-40d0-8c98-daf6818542f8 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:39.359629 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2b0d1-d193-4220-a995-f5ce2c06b987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:39.360471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2b0d1-d193-4220-a995-f5ce2c06b987 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:39.360856 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2b0d1-d193-4220-a995-f5ce2c06b987 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:39.361344 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2b0d1-d193-4220-a995-f5ce2c06b987 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:39.362384 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2b0d1-d193-4220-a995-f5ce2c06b987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:39.365870 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2b0d1-d193-4220-a995-f5ce2c06b987 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:39.386297 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1363/2721] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:39.392903 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2aad5-221a-4884-a286-21a021d7e703 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:39.410864 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d2aad5-221a-4884-a286-21a021d7e703 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:39.440037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb51f47-7359-4493-8c38-a690adc07fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:39.441020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb51f47-7359-4493-8c38-a690adc07fea admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:39.441463 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb51f47-7359-4493-8c38-a690adc07fea admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:39.441664 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb51f47-7359-4493-8c38-a690adc07fea admin admin] PATH_INFO: `/v3/projects/8118072c71ae4211aa74939e78fb4ef9/users/66a5da183c784821a2d924d9ab61b770/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:39.444953 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb51f47-7359-4493-8c38-a690adc07fea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8118072c71ae4211aa74939e78fb4ef9, user_id=66a5da183c784821a2d924d9ab61b770, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:39.448695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb51f47-7359-4493-8c38-a690adc07fea admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:39.467610 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1359/2722] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:39 2022] PUT /identity/v3/projects/8118072c71ae4211aa74939e78fb4ef9/users/66a5da183c784821a2d924d9ab61b770/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:39.491441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a36781-eb7c-43db-86d1-c53e33c3ef46 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:39.491812 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a36781-eb7c-43db-86d1-c53e33c3ef46 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:39.492709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a36781-eb7c-43db-86d1-c53e33c3ef46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:39.538149 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-f7a36781-eb7c-43db-86d1-c53e33c3ef46 None None] MFA Rules not processed for user `66a5da183c784821a2d924d9ab61b770`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:39.540617 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f7a36781-eb7c-43db-86d1-c53e33c3ef46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:39.691358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1364/2723] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:42:39.709480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb51f47-7359-4493-8c38-a690adc07fea admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:39.778743 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb51f47-7359-4493-8c38-a690adc07fea admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:39.784586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146101a5-d9a1-4d25-ac3b-faf1474c1e58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:39.785820 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146101a5-d9a1-4d25-ac3b-faf1474c1e58 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:39.786023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146101a5-d9a1-4d25-ac3b-faf1474c1e58 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:39.786337 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146101a5-d9a1-4d25-ac3b-faf1474c1e58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:39.805706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146101a5-d9a1-4d25-ac3b-faf1474c1e58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:39.808374 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146101a5-d9a1-4d25-ac3b-faf1474c1e58 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:39.827646 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1360/2724] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:42:39.938060 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a36781-eb7c-43db-86d1-c53e33c3ef46 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:39.984047 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:40.041552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a36781-eb7c-43db-86d1-c53e33c3ef46 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:40.049865 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd7357-2976-4ace-b720-03705bdbdb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:40.051004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd7357-2976-4ace-b720-03705bdbdb45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:40.051541 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd7357-2976-4ace-b720-03705bdbdb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:40.052059 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd7357-2976-4ace-b720-03705bdbdb45 admin admin] PATH_INFO: `/v3/users/f6d5fb7b2db64b509043f9bb20d8f4ad` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:40.054292 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd7357-2976-4ace-b720-03705bdbdb45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6d5fb7b2db64b509043f9bb20d8f4ad)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:40.058758 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd7357-2976-4ace-b720-03705bdbdb45 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:40.338290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-19bd7357-2976-4ace-b720-03705bdbdb45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6d5fb7b2db64b509043f9bb20d8f4ad'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:40.445775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-19bd7357-2976-4ace-b720-03705bdbdb45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6d5fb7b2db64b509043f9bb20d8f4ad'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:40.447983 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-19bd7357-2976-4ace-b720-03705bdbdb45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6d5fb7b2db64b509043f9bb20d8f4ad'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:40.448949 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-19bd7357-2976-4ace-b720-03705bdbdb45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6d5fb7b2db64b509043f9bb20d8f4ad'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:40.461697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-19bd7357-2976-4ace-b720-03705bdbdb45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6d5fb7b2db64b509043f9bb20d8f4ad'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:40.504479 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1365/2725] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:39 2022] DELETE /identity/v3/users/f6d5fb7b2db64b509043f9bb20d8f4ad => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:40.511606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18031d99-2d7b-4f23-80c9-b45bc25c1380 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:40.513202 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18031d99-2d7b-4f23-80c9-b45bc25c1380 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:40.513754 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18031d99-2d7b-4f23-80c9-b45bc25c1380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:40.547113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-18031d99-2d7b-4f23-80c9-b45bc25c1380 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:40.548674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-18031d99-2d7b-4f23-80c9-b45bc25c1380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:40.636219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1361/2726] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:40.648118 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd7357-2976-4ace-b720-03705bdbdb45 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:40.650671 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-19bd7357-2976-4ace-b720-03705bdbdb45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:40.760280 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd7357-2976-4ace-b720-03705bdbdb45 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:40.765986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27ec4a-6161-4a85-b3cf-ebf437163b2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:40.767155 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27ec4a-6161-4a85-b3cf-ebf437163b2c service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:40.767665 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27ec4a-6161-4a85-b3cf-ebf437163b2c service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:40.768175 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27ec4a-6161-4a85-b3cf-ebf437163b2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:40.786580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27ec4a-6161-4a85-b3cf-ebf437163b2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:40.788148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27ec4a-6161-4a85-b3cf-ebf437163b2c service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:40.807417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1366/2727] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:42:41.488151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18031d99-2d7b-4f23-80c9-b45bc25c1380 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:41.488151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-18031d99-2d7b-4f23-80c9-b45bc25c1380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:41.512266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18031d99-2d7b-4f23-80c9-b45bc25c1380 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:41.519543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becea101-6bee-43aa-8f21-8ff9158ad073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:41.520864 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becea101-6bee-43aa-8f21-8ff9158ad073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:41.521349 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becea101-6bee-43aa-8f21-8ff9158ad073 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:41.521830 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becea101-6bee-43aa-8f21-8ff9158ad073 admin admin] PATH_INFO: `/v3/projects/1bea16d670ff4197a13f124a9a71b9cd` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:41.524266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becea101-6bee-43aa-8f21-8ff9158ad073 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bea16d670ff4197a13f124a9a71b9cd)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:41.527916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becea101-6bee-43aa-8f21-8ff9158ad073 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:41.752455 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-becea101-6bee-43aa-8f21-8ff9158ad073 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bea16d670ff4197a13f124a9a71b9cd', 'request_id': 'req-becea101-6bee-43aa-8f21-8ff9158ad073'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:41.760353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-becea101-6bee-43aa-8f21-8ff9158ad073 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bea16d670ff4197a13f124a9a71b9cd', 'request_id': 'req-becea101-6bee-43aa-8f21-8ff9158ad073'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:41.907008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27ec4a-6161-4a85-b3cf-ebf437163b2c service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:41.908693 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1e27ec4a-6161-4a85-b3cf-ebf437163b2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:41.926100 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27ec4a-6161-4a85-b3cf-ebf437163b2c service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:41.942678 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-becea101-6bee-43aa-8f21-8ff9158ad073 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bea16d670ff4197a13f124a9a71b9cd', 'request_id': 'req-becea101-6bee-43aa-8f21-8ff9158ad073'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:41.967446 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-becea101-6bee-43aa-8f21-8ff9158ad073 admin admin] The token cache is being invalidate because project 1bea16d670ff4197a13f124a9a71b9cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:41.967961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-becea101-6bee-43aa-8f21-8ff9158ad073 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:41.976609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-becea101-6bee-43aa-8f21-8ff9158ad073 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:41.976609 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1362/2728] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:41 2022] DELETE /identity/v3/projects/1bea16d670ff4197a13f124a9a71b9cd => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:41.977982 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becea101-6bee-43aa-8f21-8ff9158ad073 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:41.980416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-becea101-6bee-43aa-8f21-8ff9158ad073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:41.997010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:42.028327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:42.041027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfeefc-7b4e-4985-b17a-db4d89dc3895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:42.042174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfeefc-7b4e-4985-b17a-db4d89dc3895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:42.042720 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfeefc-7b4e-4985-b17a-db4d89dc3895 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:42.043283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfeefc-7b4e-4985-b17a-db4d89dc3895 admin admin] PATH_INFO: `/v3/users/3f6d956add8045889ca198926b4eb7af` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:42.045682 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfeefc-7b4e-4985-b17a-db4d89dc3895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f6d956add8045889ca198926b4eb7af)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:42.049265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfeefc-7b4e-4985-b17a-db4d89dc3895 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:42.103339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becea101-6bee-43aa-8f21-8ff9158ad073 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:42.112438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167e886-739f-4c26-993c-ac324397b3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:42.113649 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e886-739f-4c26-993c-ac324397b3fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:42.114235 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e886-739f-4c26-993c-ac324397b3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:42.114789 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e886-739f-4c26-993c-ac324397b3fd admin admin] PATH_INFO: `/v3/users/66a5da183c784821a2d924d9ab61b770` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:42.118784 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167e886-739f-4c26-993c-ac324397b3fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=66a5da183c784821a2d924d9ab61b770)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:42.126183 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167e886-739f-4c26-993c-ac324397b3fd admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:42.388521 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-4ebfeefc-7b4e-4985-b17a-db4d89dc3895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f6d956add8045889ca198926b4eb7af'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:42.500587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a167e886-739f-4c26-993c-ac324397b3fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66a5da183c784821a2d924d9ab61b770'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:42.520636 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-4ebfeefc-7b4e-4985-b17a-db4d89dc3895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f6d956add8045889ca198926b4eb7af'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:42.520957 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a167e886-739f-4c26-993c-ac324397b3fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66a5da183c784821a2d924d9ab61b770'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:42.522042 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a167e886-739f-4c26-993c-ac324397b3fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66a5da183c784821a2d924d9ab61b770'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:42.522718 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-4ebfeefc-7b4e-4985-b17a-db4d89dc3895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f6d956add8045889ca198926b4eb7af'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:42.524641 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-4ebfeefc-7b4e-4985-b17a-db4d89dc3895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f6d956add8045889ca198926b4eb7af'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:42.536598 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-4ebfeefc-7b4e-4985-b17a-db4d89dc3895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f6d956add8045889ca198926b4eb7af'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:42.542661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a167e886-739f-4c26-993c-ac324397b3fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66a5da183c784821a2d924d9ab61b770'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:42.544251 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-a167e886-739f-4c26-993c-ac324397b3fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66a5da183c784821a2d924d9ab61b770'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:42.567615 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1367/2729] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:41 2022] DELETE /identity/v3/users/3f6d956add8045889ca198926b4eb7af => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:42.569137 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfeefc-7b4e-4985-b17a-db4d89dc3895 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:42.571656 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-4ebfeefc-7b4e-4985-b17a-db4d89dc3895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:42.669756 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1363/2730] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:41 2022] DELETE /identity/v3/users/66a5da183c784821a2d924d9ab61b770 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:42.670902 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9098f7c-3417-4c04-94a3-24776b972e97 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:42.671144 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9098f7c-3417-4c04-94a3-24776b972e97 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:42.671608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9098f7c-3417-4c04-94a3-24776b972e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:42.699869 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-e9098f7c-3417-4c04-94a3-24776b972e97 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:42.702630 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e9098f7c-3417-4c04-94a3-24776b972e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:42.719601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfeefc-7b4e-4985-b17a-db4d89dc3895 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:42.828821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629559ce-fd71-42ea-8d73-f8de0f34ff80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:42.831986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629559ce-fd71-42ea-8d73-f8de0f34ff80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:42.831986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629559ce-fd71-42ea-8d73-f8de0f34ff80 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:42.831986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629559ce-fd71-42ea-8d73-f8de0f34ff80 admin admin] PATH_INFO: `/v3/users/d8cd82c577384a4b881bea09cb317535` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:42.832827 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629559ce-fd71-42ea-8d73-f8de0f34ff80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8cd82c577384a4b881bea09cb317535)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:42.835354 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629559ce-fd71-42ea-8d73-f8de0f34ff80 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:42.859567 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1364/2731] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:42.872351 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9098f7c-3417-4c04-94a3-24776b972e97 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:42.875336 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e9098f7c-3417-4c04-94a3-24776b972e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:42.889586 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:42.929552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:42.989243 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9098f7c-3417-4c04-94a3-24776b972e97 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:42.997496 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a56eb-7c60-4602-9e91-fda05ee82d24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:42.999025 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a56eb-7c60-4602-9e91-fda05ee82d24 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:42.999825 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a56eb-7c60-4602-9e91-fda05ee82d24 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:43.000536 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a56eb-7c60-4602-9e91-fda05ee82d24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:43.021181 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a56eb-7c60-4602-9e91-fda05ee82d24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:43.029484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a56eb-7c60-4602-9e91-fda05ee82d24 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:43.043853 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1365/2732] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:42:43.210118 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-629559ce-fd71-42ea-8d73-f8de0f34ff80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8cd82c577384a4b881bea09cb317535'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:43.377125 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-629559ce-fd71-42ea-8d73-f8de0f34ff80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8cd82c577384a4b881bea09cb317535'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:43.378248 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-629559ce-fd71-42ea-8d73-f8de0f34ff80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8cd82c577384a4b881bea09cb317535'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:43.379362 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-629559ce-fd71-42ea-8d73-f8de0f34ff80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8cd82c577384a4b881bea09cb317535'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:43.396243 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a56eb-7c60-4602-9e91-fda05ee82d24 service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:43.399039 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-629559ce-fd71-42ea-8d73-f8de0f34ff80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8cd82c577384a4b881bea09cb317535'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:43.409323 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-335a56eb-7c60-4602-9e91-fda05ee82d24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:43.423736 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a56eb-7c60-4602-9e91-fda05ee82d24 service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:43.429060 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4b506-5715-4571-b434-641bcd35b33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:43.430739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4b506-5715-4571-b434-641bcd35b33b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:43.431103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4b506-5715-4571-b434-641bcd35b33b admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:43.431481 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4b506-5715-4571-b434-641bcd35b33b admin admin] PATH_INFO: `/v3/projects/8118072c71ae4211aa74939e78fb4ef9` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:43.433704 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4b506-5715-4571-b434-641bcd35b33b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8118072c71ae4211aa74939e78fb4ef9)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:43.437616 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4b506-5715-4571-b434-641bcd35b33b admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:43.438914 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1368/2733] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:42 2022] DELETE /identity/v3/users/d8cd82c577384a4b881bea09cb317535 => generated 0 bytes in 866 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:43.441248 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a26826-66c8-4ad9-8765-d4e238364dfc None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:43.441693 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a26826-66c8-4ad9-8765-d4e238364dfc None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:43.442202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a26826-66c8-4ad9-8765-d4e238364dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:43.484920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-11a26826-66c8-4ad9-8765-d4e238364dfc None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:43.487986 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-11a26826-66c8-4ad9-8765-d4e238364dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:43.626830 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1369/2734] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:43.643265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a26826-66c8-4ad9-8765-d4e238364dfc None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:43.644909 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-11a26826-66c8-4ad9-8765-d4e238364dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:43.654651 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-97e4b506-5715-4571-b434-641bcd35b33b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8118072c71ae4211aa74939e78fb4ef9', 'request_id': 'req-97e4b506-5715-4571-b434-641bcd35b33b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:43.674131 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-97e4b506-5715-4571-b434-641bcd35b33b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8118072c71ae4211aa74939e78fb4ef9', 'request_id': 'req-97e4b506-5715-4571-b434-641bcd35b33b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:43.752907 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a26826-66c8-4ad9-8765-d4e238364dfc None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:43.754546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-97e4b506-5715-4571-b434-641bcd35b33b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8118072c71ae4211aa74939e78fb4ef9', 'request_id': 'req-97e4b506-5715-4571-b434-641bcd35b33b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:43.779635 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-97e4b506-5715-4571-b434-641bcd35b33b admin admin] The token cache is being invalidate because project 8118072c71ae4211aa74939e78fb4ef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:43.780119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-97e4b506-5715-4571-b434-641bcd35b33b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:43.781203 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-97e4b506-5715-4571-b434-641bcd35b33b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:43.783730 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1366/2735] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:43 2022] DELETE /identity/v3/projects/8118072c71ae4211aa74939e78fb4ef9 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:43.784997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4b506-5715-4571-b434-641bcd35b33b admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:43.787946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-97e4b506-5715-4571-b434-641bcd35b33b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:43.848153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eef5e4-f7c7-4b33-9ec9-9623c58a6c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:43.849231 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eef5e4-f7c7-4b33-9ec9-9623c58a6c2f service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:43.849707 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eef5e4-f7c7-4b33-9ec9-9623c58a6c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:43.850093 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eef5e4-f7c7-4b33-9ec9-9623c58a6c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:43.854736 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e2eef5e4-f7c7-4b33-9ec9-9623c58a6c2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:43.877918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:43.907401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4b506-5715-4571-b434-641bcd35b33b admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:43.913357 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd37c6-7d78-4e02-90ab-38d347958a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:43.915223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd37c6-7d78-4e02-90ab-38d347958a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:43.915586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd37c6-7d78-4e02-90ab-38d347958a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:43.915946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd37c6-7d78-4e02-90ab-38d347958a49 admin admin] PATH_INFO: `/v3/projects/f2068ee0e18a44e0b3e4d568b7d8ed15` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:43.917351 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd37c6-7d78-4e02-90ab-38d347958a49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2068ee0e18a44e0b3e4d568b7d8ed15)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:43.919532 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd37c6-7d78-4e02-90ab-38d347958a49 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:43.967374 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eef5e4-f7c7-4b33-9ec9-9623c58a6c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:43.969162 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eef5e4-f7c7-4b33-9ec9-9623c58a6c2f service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:43.989073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1370/2736] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:42:43.989073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339f119-fe7c-4495-a7d6-3ca779e7e0c0 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:43.989448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339f119-fe7c-4495-a7d6-3ca779e7e0c0 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:43.989448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339f119-fe7c-4495-a7d6-3ca779e7e0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:44.012879 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-c339f119-fe7c-4495-a7d6-3ca779e7e0c0 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:44.014216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c339f119-fe7c-4495-a7d6-3ca779e7e0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:44.027500 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1371/2737] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:44.028652 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339f119-fe7c-4495-a7d6-3ca779e7e0c0 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:44.030717 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c339f119-fe7c-4495-a7d6-3ca779e7e0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:44.047583 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339f119-fe7c-4495-a7d6-3ca779e7e0c0 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:44.055175 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6633649-20ac-4b37-83db-367fb346ff5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:44.056445 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6633649-20ac-4b37-83db-367fb346ff5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:44.057585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6633649-20ac-4b37-83db-367fb346ff5d admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:44.057585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6633649-20ac-4b37-83db-367fb346ff5d admin admin] PATH_INFO: `/v3/users/9aec64f63f5b4e09bcd02ab62444057d` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:44.059692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6633649-20ac-4b37-83db-367fb346ff5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aec64f63f5b4e09bcd02ab62444057d)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:44.064186 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6633649-20ac-4b37-83db-367fb346ff5d admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:44.097948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-18fd37c6-7d78-4e02-90ab-38d347958a49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2068ee0e18a44e0b3e4d568b7d8ed15', 'request_id': 'req-18fd37c6-7d78-4e02-90ab-38d347958a49'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.102919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:44.110537 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-18fd37c6-7d78-4e02-90ab-38d347958a49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2068ee0e18a44e0b3e4d568b7d8ed15', 'request_id': 'req-18fd37c6-7d78-4e02-90ab-38d347958a49'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.216875 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-18fd37c6-7d78-4e02-90ab-38d347958a49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2068ee0e18a44e0b3e4d568b7d8ed15', 'request_id': 'req-18fd37c6-7d78-4e02-90ab-38d347958a49'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.229210 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-18fd37c6-7d78-4e02-90ab-38d347958a49 admin admin] The token cache is being invalidate because project f2068ee0e18a44e0b3e4d568b7d8ed15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:44.229578 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-18fd37c6-7d78-4e02-90ab-38d347958a49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.230191 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-18fd37c6-7d78-4e02-90ab-38d347958a49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.232120 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1367/2738] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:43 2022] DELETE /identity/v3/projects/f2068ee0e18a44e0b3e4d568b7d8ed15 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:44.236624 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae7e8f-8544-4694-b8bb-26271dbf54a1 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:44.236624 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae7e8f-8544-4694-b8bb-26271dbf54a1 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:44.236914 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae7e8f-8544-4694-b8bb-26271dbf54a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:44.270135 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-04ae7e8f-8544-4694-b8bb-26271dbf54a1 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:44.271468 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-04ae7e8f-8544-4694-b8bb-26271dbf54a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:44.345762 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e6633649-20ac-4b37-83db-367fb346ff5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aec64f63f5b4e09bcd02ab62444057d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.384351 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1368/2739] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:44.385466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae7e8f-8544-4694-b8bb-26271dbf54a1 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:44.387448 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-04ae7e8f-8544-4694-b8bb-26271dbf54a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:44.405467 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae7e8f-8544-4694-b8bb-26271dbf54a1 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:44.412849 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616775f6-8b58-4a79-8f30-17d00c57dbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:44.414507 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616775f6-8b58-4a79-8f30-17d00c57dbf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:44.415016 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616775f6-8b58-4a79-8f30-17d00c57dbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:44.417785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616775f6-8b58-4a79-8f30-17d00c57dbf4 admin admin] PATH_INFO: `/v3/users/634d91a206a947b691f42befd1737b2b` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:44.417785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616775f6-8b58-4a79-8f30-17d00c57dbf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=634d91a206a947b691f42befd1737b2b)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:44.419516 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616775f6-8b58-4a79-8f30-17d00c57dbf4 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:44.441860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e6633649-20ac-4b37-83db-367fb346ff5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aec64f63f5b4e09bcd02ab62444057d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.443036 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e6633649-20ac-4b37-83db-367fb346ff5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aec64f63f5b4e09bcd02ab62444057d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.449234 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e6633649-20ac-4b37-83db-367fb346ff5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aec64f63f5b4e09bcd02ab62444057d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.456350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e6633649-20ac-4b37-83db-367fb346ff5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aec64f63f5b4e09bcd02ab62444057d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.485972 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1372/2740] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:44 2022] DELETE /identity/v3/users/9aec64f63f5b4e09bcd02ab62444057d => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:44.486860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6633649-20ac-4b37-83db-367fb346ff5d admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:44.488395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e6633649-20ac-4b37-83db-367fb346ff5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:44.580486 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6633649-20ac-4b37-83db-367fb346ff5d admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:44.589000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc047ea-3380-422c-a8b3-94260501277f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:44.589811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc047ea-3380-422c-a8b3-94260501277f admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:44.590273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc047ea-3380-422c-a8b3-94260501277f admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:44.590783 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc047ea-3380-422c-a8b3-94260501277f admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:44.592148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc047ea-3380-422c-a8b3-94260501277f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:42:44.592370 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc047ea-3380-422c-a8b3-94260501277f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:44.594939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc047ea-3380-422c-a8b3-94260501277f admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:44.611440 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1373/2741] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:42:44.613417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f089e-c786-445e-be70-e2c4341a6303 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:44.613905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f089e-c786-445e-be70-e2c4341a6303 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:44.614247 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f089e-c786-445e-be70-e2c4341a6303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:44.633333 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:44.641400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-9a4f089e-c786-445e-be70-e2c4341a6303 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:44.642944 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9a4f089e-c786-445e-be70-e2c4341a6303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:44.653843 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1374/2742] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:44.654944 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f089e-c786-445e-be70-e2c4341a6303 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:44.656960 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9a4f089e-c786-445e-be70-e2c4341a6303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:44.670655 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f089e-c786-445e-be70-e2c4341a6303 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:44.675616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfebb6-de9b-49aa-9905-1ba265bbf691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:44.676265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfebb6-de9b-49aa-9905-1ba265bbf691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:44.676621 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfebb6-de9b-49aa-9905-1ba265bbf691 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:44.676892 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfebb6-de9b-49aa-9905-1ba265bbf691 admin admin] PATH_INFO: `/v3/projects/4eef4acdfbfe417493bcedcdc78b0f69` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:44.678168 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfebb6-de9b-49aa-9905-1ba265bbf691 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eef4acdfbfe417493bcedcdc78b0f69)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:44.680457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfebb6-de9b-49aa-9905-1ba265bbf691 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:44.712796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-616775f6-8b58-4a79-8f30-17d00c57dbf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '634d91a206a947b691f42befd1737b2b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.729014 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-616775f6-8b58-4a79-8f30-17d00c57dbf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '634d91a206a947b691f42befd1737b2b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.729796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-616775f6-8b58-4a79-8f30-17d00c57dbf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '634d91a206a947b691f42befd1737b2b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.738556 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-616775f6-8b58-4a79-8f30-17d00c57dbf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '634d91a206a947b691f42befd1737b2b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.739310 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-616775f6-8b58-4a79-8f30-17d00c57dbf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '634d91a206a947b691f42befd1737b2b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.832350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-72cfebb6-de9b-49aa-9905-1ba265bbf691 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eef4acdfbfe417493bcedcdc78b0f69', 'request_id': 'req-72cfebb6-de9b-49aa-9905-1ba265bbf691'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.841070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-72cfebb6-de9b-49aa-9905-1ba265bbf691 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eef4acdfbfe417493bcedcdc78b0f69', 'request_id': 'req-72cfebb6-de9b-49aa-9905-1ba265bbf691'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.851442 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1369/2743] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:44 2022] DELETE /identity/v3/users/634d91a206a947b691f42befd1737b2b => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:44.853339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616775f6-8b58-4a79-8f30-17d00c57dbf4 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:44.855907 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-616775f6-8b58-4a79-8f30-17d00c57dbf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:44.954306 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:44.971056 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-72cfebb6-de9b-49aa-9905-1ba265bbf691 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eef4acdfbfe417493bcedcdc78b0f69', 'request_id': 'req-72cfebb6-de9b-49aa-9905-1ba265bbf691'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.990226 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-72cfebb6-de9b-49aa-9905-1ba265bbf691 admin admin] The token cache is being invalidate because project 4eef4acdfbfe417493bcedcdc78b0f69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:44.990708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-72cfebb6-de9b-49aa-9905-1ba265bbf691 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.992518 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-72cfebb6-de9b-49aa-9905-1ba265bbf691 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:44.996051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616775f6-8b58-4a79-8f30-17d00c57dbf4 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:44.996465 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1375/2744] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:44 2022] DELETE /identity/v3/projects/4eef4acdfbfe417493bcedcdc78b0f69 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:45.003775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfebb6-de9b-49aa-9905-1ba265bbf691 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:45.003775 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-72cfebb6-de9b-49aa-9905-1ba265bbf691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:45.024364 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5eef1f-c162-43b4-8da1-8d8ae9c84c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:45.025376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eef1f-c162-43b4-8da1-8d8ae9c84c4c admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:45.025851 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eef1f-c162-43b4-8da1-8d8ae9c84c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:45.026367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eef1f-c162-43b4-8da1-8d8ae9c84c4c admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:45.027502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5eef1f-c162-43b4-8da1-8d8ae9c84c4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:45.030561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5eef1f-c162-43b4-8da1-8d8ae9c84c4c admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:45.103735 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfebb6-de9b-49aa-9905-1ba265bbf691 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:45.175210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b31fc-5766-4cc8-bd31-4f3dd68687aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:45.175841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b31fc-5766-4cc8-bd31-4f3dd68687aa service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:45.176123 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b31fc-5766-4cc8-bd31-4f3dd68687aa service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:45.176385 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b31fc-5766-4cc8-bd31-4f3dd68687aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:45.178262 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-bf8b31fc-5766-4cc8-bd31-4f3dd68687aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:45.192359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b31fc-5766-4cc8-bd31-4f3dd68687aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:45.196667 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b31fc-5766-4cc8-bd31-4f3dd68687aa service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:45.205489 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1370/2745] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:44 2022] POST /identity/v3/projects => generated 391 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:45.207449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38afd423-3070-4123-95c5-58e8664ca6d4 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:45.207801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38afd423-3070-4123-95c5-58e8664ca6d4 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:45.208047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38afd423-3070-4123-95c5-58e8664ca6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:45.208210 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1376/2746] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:42:45.210004 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b31fc-5766-4cc8-bd31-4f3dd68687aa service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:45.213465 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-bf8b31fc-5766-4cc8-bd31-4f3dd68687aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:45.235747 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-38afd423-3070-4123-95c5-58e8664ca6d4 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:45.237294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-38afd423-3070-4123-95c5-58e8664ca6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:45.244113 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b31fc-5766-4cc8-bd31-4f3dd68687aa service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:45.370015 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:45.387218 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1371/2747] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:45.393014 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cc5f6-3065-41bb-96b8-94f3216f8dc5 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:45.393345 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cc5f6-3065-41bb-96b8-94f3216f8dc5 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:45.393576 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cc5f6-3065-41bb-96b8-94f3216f8dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:45.417970 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fa93d-9d1f-48d6-b37e-b9ac0c448247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:45.419596 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fa93d-9d1f-48d6-b37e-b9ac0c448247 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:45.420156 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fa93d-9d1f-48d6-b37e-b9ac0c448247 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:45.420862 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001fa93d-9d1f-48d6-b37e-b9ac0c448247 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:45.422009 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001fa93d-9d1f-48d6-b37e-b9ac0c448247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:45.424128 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-e57cc5f6-3065-41bb-96b8-94f3216f8dc5 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:45.425371 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e57cc5f6-3065-41bb-96b8-94f3216f8dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:45.426233 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001fa93d-9d1f-48d6-b37e-b9ac0c448247 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:45.436616 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1372/2748] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:42:45.442356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cc5f6-3065-41bb-96b8-94f3216f8dc5 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:45.457650 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cc5f6-3065-41bb-96b8-94f3216f8dc5 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:45.465355 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1c615-3be2-478c-8a69-836fd13d7380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:45.465653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1c615-3be2-478c-8a69-836fd13d7380 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:45.465838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1c615-3be2-478c-8a69-836fd13d7380 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:45.466051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1c615-3be2-478c-8a69-836fd13d7380 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:45.466822 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1c615-3be2-478c-8a69-836fd13d7380 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:42:45.475015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1c615-3be2-478c-8a69-836fd13d7380 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:45.475015 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1c615-3be2-478c-8a69-836fd13d7380 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:45.487031 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1373/2749] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:42:45.491061 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1c615-3be2-478c-8a69-836fd13d7380 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:45.505144 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1c615-3be2-478c-8a69-836fd13d7380 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:45.511449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7d657-153c-4b56-a530-4615a2e33eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:45.512572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7d657-153c-4b56-a530-4615a2e33eda admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:45.512572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7d657-153c-4b56-a530-4615a2e33eda admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:45.512918 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7d657-153c-4b56-a530-4615a2e33eda admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:45.513285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7d657-153c-4b56-a530-4615a2e33eda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:45.515322 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7d657-153c-4b56-a530-4615a2e33eda admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:45.528743 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1377/2750] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:45 2022] POST /identity/v3/users => generated 296 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:45.535541 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fa93d-9d1f-48d6-b37e-b9ac0c448247 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:45.554400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001fa93d-9d1f-48d6-b37e-b9ac0c448247 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:45.561703 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0c1cb-2563-4ea8-a9a5-9632b89f735d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:45.562631 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0c1cb-2563-4ea8-a9a5-9632b89f735d admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:45.563022 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0c1cb-2563-4ea8-a9a5-9632b89f735d admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:45.563478 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0c1cb-2563-4ea8-a9a5-9632b89f735d admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:45.564649 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0c1cb-2563-4ea8-a9a5-9632b89f735d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:45.566830 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0c1cb-2563-4ea8-a9a5-9632b89f735d admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:45.581264 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1378/2751] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:45.587317 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0c1cb-2563-4ea8-a9a5-9632b89f735d admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:45.599136 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0c1cb-2563-4ea8-a9a5-9632b89f735d admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:45.606648 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8dbff-c50d-43bc-825f-1c4b0bdb1b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:45.607803 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8dbff-c50d-43bc-825f-1c4b0bdb1b26 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:45.608408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8dbff-c50d-43bc-825f-1c4b0bdb1b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:45.608990 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8dbff-c50d-43bc-825f-1c4b0bdb1b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:45.610279 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8dbff-c50d-43bc-825f-1c4b0bdb1b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:45.611969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8dbff-c50d-43bc-825f-1c4b0bdb1b26 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:45.629412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1379/2752] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:42:45.629412 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8dbff-c50d-43bc-825f-1c4b0bdb1b26 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:45.629780 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9aa8dbff-c50d-43bc-825f-1c4b0bdb1b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:45.646427 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8dbff-c50d-43bc-825f-1c4b0bdb1b26 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:45.648361 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1374/2753] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:45 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:42:45.656158 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7d657-153c-4b56-a530-4615a2e33eda admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:45.748120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:45.748446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:45.748677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:45.748870 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd admin admin] PATH_INFO: `/v3/projects/6f5dbf4c6ae1426fa94302b26e23062b` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:45.750164 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f5dbf4c6ae1426fa94302b26e23062b)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:45.752728 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:45.770777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7d657-153c-4b56-a530-4615a2e33eda admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:45.780901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40164a25-bca5-4950-b6c7-5392f9a2f8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:45.782430 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40164a25-bca5-4950-b6c7-5392f9a2f8da admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:45.782938 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40164a25-bca5-4950-b6c7-5392f9a2f8da admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:45.785837 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40164a25-bca5-4950-b6c7-5392f9a2f8da admin admin] PATH_INFO: `/v3/projects/cc1f99ce86c4420ca9afc560cc5a3155/users/2abc4cab476d47918f16ec53a8106376/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:45.816322 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40164a25-bca5-4950-b6c7-5392f9a2f8da admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1f99ce86c4420ca9afc560cc5a3155, user_id=2abc4cab476d47918f16ec53a8106376, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:45.819184 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40164a25-bca5-4950-b6c7-5392f9a2f8da admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:45.908510 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1375/2754] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:45 2022] PUT /identity/v3/projects/cc1f99ce86c4420ca9afc560cc5a3155/users/2abc4cab476d47918f16ec53a8106376/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:45.909118 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40164a25-bca5-4950-b6c7-5392f9a2f8da admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:45.931929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f5dbf4c6ae1426fa94302b26e23062b', 'request_id': 'req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:45.935778 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:45.944134 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f5dbf4c6ae1426fa94302b26e23062b', 'request_id': 'req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:45.984379 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40164a25-bca5-4950-b6c7-5392f9a2f8da admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:46.029395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f5dbf4c6ae1426fa94302b26e23062b', 'request_id': 'req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:46.046211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd admin admin] The token cache is being invalidate because project 6f5dbf4c6ae1426fa94302b26e23062b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:46.046579 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:46.047388 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:46.049045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1380/2755] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:45 2022] DELETE /identity/v3/projects/6f5dbf4c6ae1426fa94302b26e23062b => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:46.056091 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:46.057771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:46.059866 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4193dd-0e78-479e-a97d-678dbb8c76e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:46.067353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4193dd-0e78-479e-a97d-678dbb8c76e6 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:46.067353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4193dd-0e78-479e-a97d-678dbb8c76e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:46.067353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4193dd-0e78-479e-a97d-678dbb8c76e6 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:46.067353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4193dd-0e78-479e-a97d-678dbb8c76e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:46.067353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4193dd-0e78-479e-a97d-678dbb8c76e6 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:46.073975 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfa3fe-0f8a-46db-b5e7-bad6e59215fd admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:46.079236 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:46.083246 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:46.083246 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:46.083246 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321 admin admin] PATH_INFO: `/v3/projects/4660fe0f55ef4c7cba4ee7374d2422ee` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:46.083246 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4660fe0f55ef4c7cba4ee7374d2422ee)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:46.083747 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:46.226613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1376/2756] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:45 2022] POST /identity/v3/users => generated 292 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:46.229038 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58fa20-3aad-4d2a-887a-058f545cbed4 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:46.229645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58fa20-3aad-4d2a-887a-058f545cbed4 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:46.230155 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58fa20-3aad-4d2a-887a-058f545cbed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:46.241463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4660fe0f55ef4c7cba4ee7374d2422ee', 'request_id': 'req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:46.252370 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4660fe0f55ef4c7cba4ee7374d2422ee', 'request_id': 'req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:46.271866 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-2d58fa20-3aad-4d2a-887a-058f545cbed4 None None] MFA Rules not processed for user `2abc4cab476d47918f16ec53a8106376`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:46.279178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-2d58fa20-3aad-4d2a-887a-058f545cbed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:46.357521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1377/2757] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:46.358845 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984efb88-af2c-4da7-af91-954211b6fc3a None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:46.359171 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984efb88-af2c-4da7-af91-954211b6fc3a None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:46.359457 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984efb88-af2c-4da7-af91-954211b6fc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:46.371042 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4660fe0f55ef4c7cba4ee7374d2422ee', 'request_id': 'req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:46.383139 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-984efb88-af2c-4da7-af91-954211b6fc3a None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:46.384744 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-984efb88-af2c-4da7-af91-954211b6fc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:46.388255 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321 admin admin] The token cache is being invalidate because project 4660fe0f55ef4c7cba4ee7374d2422ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:46.388639 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:46.389506 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:46.391236 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1381/2758] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:46 2022] DELETE /identity/v3/projects/4660fe0f55ef4c7cba4ee7374d2422ee => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:46.395800 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:46.397569 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:46.473027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1378/2759] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:46.474106 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984efb88-af2c-4da7-af91-954211b6fc3a None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:46.481692 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-984efb88-af2c-4da7-af91-954211b6fc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:46.491259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87edfd-3e6c-49e0-bbbc-dc763af2a321 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:46.499282 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882dfef-0b41-4559-9347-e2230d9bb016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:46.499950 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882dfef-0b41-4559-9347-e2230d9bb016 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:46.500585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882dfef-0b41-4559-9347-e2230d9bb016 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:46.500864 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882dfef-0b41-4559-9347-e2230d9bb016 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:46.501556 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882dfef-0b41-4559-9347-e2230d9bb016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:46.501766 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984efb88-af2c-4da7-af91-954211b6fc3a None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:46.503235 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882dfef-0b41-4559-9347-e2230d9bb016 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:46.510181 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db7269-c7cd-45a2-98c7-e29e8b8adeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:46.511227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db7269-c7cd-45a2-98c7-e29e8b8adeb2 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:46.511657 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db7269-c7cd-45a2-98c7-e29e8b8adeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:46.512076 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db7269-c7cd-45a2-98c7-e29e8b8adeb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:46.513012 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db7269-c7cd-45a2-98c7-e29e8b8adeb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:46.515708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1382/2760] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:46.516626 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882dfef-0b41-4559-9347-e2230d9bb016 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:46.518915 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7882dfef-0b41-4559-9347-e2230d9bb016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:46.521047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db7269-c7cd-45a2-98c7-e29e8b8adeb2 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:46.534426 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882dfef-0b41-4559-9347-e2230d9bb016 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:46.540475 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f1493-4af1-44b4-ad0b-f2e7f6822554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:46.541147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f1493-4af1-44b4-ad0b-f2e7f6822554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:46.541422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f1493-4af1-44b4-ad0b-f2e7f6822554 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:46.541696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f1493-4af1-44b4-ad0b-f2e7f6822554 admin admin] PATH_INFO: `/v3/users/cb55a5a35b8a479fbc1ba3d1d284ff1d` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:46.543027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f1493-4af1-44b4-ad0b-f2e7f6822554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb55a5a35b8a479fbc1ba3d1d284ff1d)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:46.545000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f1493-4af1-44b4-ad0b-f2e7f6822554 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:46.644444 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1379/2761] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:46 2022] POST /identity/v3/projects => generated 391 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:46.644444 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7861e7-8b19-4fbf-bc79-2764b4a96304 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:46.644444 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7861e7-8b19-4fbf-bc79-2764b4a96304 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:46.644444 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7861e7-8b19-4fbf-bc79-2764b4a96304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:46.666287 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-1f7861e7-8b19-4fbf-bc79-2764b4a96304 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:46.667577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-1f7861e7-8b19-4fbf-bc79-2764b4a96304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:46.741604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1380/2762] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:46.741604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7861e7-8b19-4fbf-bc79-2764b4a96304 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:46.750270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7861e7-8b19-4fbf-bc79-2764b4a96304 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:46.755215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b23f7-8204-4803-b749-336474889154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:46.755856 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b23f7-8204-4803-b749-336474889154 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:46.756153 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b23f7-8204-4803-b749-336474889154 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:46.756411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b23f7-8204-4803-b749-336474889154 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:46.757062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b23f7-8204-4803-b749-336474889154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:46.759369 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b23f7-8204-4803-b749-336474889154 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:46.771205 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1381/2763] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:46.772191 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b23f7-8204-4803-b749-336474889154 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:46.781940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-531f1493-4af1-44b4-ad0b-f2e7f6822554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb55a5a35b8a479fbc1ba3d1d284ff1d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:46.789690 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b23f7-8204-4803-b749-336474889154 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:46.795730 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2329ce5-6b63-4c2c-9850-e56ec2244a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:46.796357 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2329ce5-6b63-4c2c-9850-e56ec2244a7f admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:46.796723 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2329ce5-6b63-4c2c-9850-e56ec2244a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:46.797104 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2329ce5-6b63-4c2c-9850-e56ec2244a7f admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:46.797941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2329ce5-6b63-4c2c-9850-e56ec2244a7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:46.803129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2329ce5-6b63-4c2c-9850-e56ec2244a7f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:46.924487 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-531f1493-4af1-44b4-ad0b-f2e7f6822554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb55a5a35b8a479fbc1ba3d1d284ff1d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:46.925162 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-531f1493-4af1-44b4-ad0b-f2e7f6822554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb55a5a35b8a479fbc1ba3d1d284ff1d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:46.925791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-531f1493-4af1-44b4-ad0b-f2e7f6822554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb55a5a35b8a479fbc1ba3d1d284ff1d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:46.933904 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1382/2764] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:46 2022] POST /identity/v3/users => generated 302 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:46.935479 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2329ce5-6b63-4c2c-9850-e56ec2244a7f admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:46.938106 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a2329ce5-6b63-4c2c-9850-e56ec2244a7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:46.939814 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-531f1493-4af1-44b4-ad0b-f2e7f6822554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb55a5a35b8a479fbc1ba3d1d284ff1d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:46.953540 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:46.958343 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2329ce5-6b63-4c2c-9850-e56ec2244a7f admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:46.965985 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd59b49-001a-4f47-8c99-0f0858af2044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:46.966859 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd59b49-001a-4f47-8c99-0f0858af2044 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:46.967298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd59b49-001a-4f47-8c99-0f0858af2044 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:46.967705 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd59b49-001a-4f47-8c99-0f0858af2044 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:46.968781 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd59b49-001a-4f47-8c99-0f0858af2044 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:42:46.969269 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd59b49-001a-4f47-8c99-0f0858af2044 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:46.974483 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1383/2765] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:46 2022] DELETE /identity/v3/users/cb55a5a35b8a479fbc1ba3d1d284ff1d => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:46.974723 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd59b49-001a-4f47-8c99-0f0858af2044 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:46.974901 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f1493-4af1-44b4-ad0b-f2e7f6822554 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:46.974901 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-531f1493-4af1-44b4-ad0b-f2e7f6822554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:46.987094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1383/2766] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:42:46.988522 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd59b49-001a-4f47-8c99-0f0858af2044 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:46.995550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ddd59b49-001a-4f47-8c99-0f0858af2044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:47.010356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd59b49-001a-4f47-8c99-0f0858af2044 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:47.077074 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f1493-4af1-44b4-ad0b-f2e7f6822554 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:47.082721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:47.090926 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f58a7-37be-4946-b537-1e8f363b6917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:47.092187 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f58a7-37be-4946-b537-1e8f363b6917 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:47.092669 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f58a7-37be-4946-b537-1e8f363b6917 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:47.093153 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f58a7-37be-4946-b537-1e8f363b6917 admin admin] PATH_INFO: `/v3/projects/b933078b01aa4e3b82e37fef9b6ff92c/users/8895cee0f4444866bd4c3908eb248543/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:47.099978 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653c738-4fa5-4c23-a1be-1017d0e8a0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:47.099978 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653c738-4fa5-4c23-a1be-1017d0e8a0f0 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:47.099978 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653c738-4fa5-4c23-a1be-1017d0e8a0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:47.099978 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653c738-4fa5-4c23-a1be-1017d0e8a0f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:47.100807 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f653c738-4fa5-4c23-a1be-1017d0e8a0f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:47.103317 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f653c738-4fa5-4c23-a1be-1017d0e8a0f0 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:47.115934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1384/2767] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:47.116587 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653c738-4fa5-4c23-a1be-1017d0e8a0f0 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:47.128423 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653c738-4fa5-4c23-a1be-1017d0e8a0f0 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:47.130695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f58a7-37be-4946-b537-1e8f363b6917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b933078b01aa4e3b82e37fef9b6ff92c, user_id=8895cee0f4444866bd4c3908eb248543, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:47.134108 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019096a-3e23-41c8-b5e9-9dad68ca2596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:47.135020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019096a-3e23-41c8-b5e9-9dad68ca2596 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:47.135362 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019096a-3e23-41c8-b5e9-9dad68ca2596 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:47.135647 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f58a7-37be-4946-b537-1e8f363b6917 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:47.135761 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019096a-3e23-41c8-b5e9-9dad68ca2596 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:47.136438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019096a-3e23-41c8-b5e9-9dad68ca2596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:47.138571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019096a-3e23-41c8-b5e9-9dad68ca2596 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:47.251518 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1384/2768] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:46 2022] PUT /identity/v3/projects/b933078b01aa4e3b82e37fef9b6ff92c/users/8895cee0f4444866bd4c3908eb248543/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:47.252761 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f58a7-37be-4946-b537-1e8f363b6917 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:47.262906 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1385/2769] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:47 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:47.265798 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aaaf87-373d-4d37-a5b2-a59315b720f8 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:47.266372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aaaf87-373d-4d37-a5b2-a59315b720f8 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:47.266864 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aaaf87-373d-4d37-a5b2-a59315b720f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:47.308068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-d5aaaf87-373d-4d37-a5b2-a59315b720f8 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:47.310286 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d5aaaf87-373d-4d37-a5b2-a59315b720f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:47.375552 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f58a7-37be-4946-b537-1e8f363b6917 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:47.424769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1386/2770] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:47.429721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aaaf87-373d-4d37-a5b2-a59315b720f8 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:47.452737 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aaaf87-373d-4d37-a5b2-a59315b720f8 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:47.456874 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37f86d-6c53-4f02-94cf-217ba111f064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:47.457554 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37f86d-6c53-4f02-94cf-217ba111f064 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:47.457831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37f86d-6c53-4f02-94cf-217ba111f064 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:47.458103 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37f86d-6c53-4f02-94cf-217ba111f064 admin admin] PATH_INFO: `/v3/projects/79c19d0118be4a2a943f1f8b80c2ee6e/users/5bf472768a2c40fba7fef8466253127a/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:47.462125 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34322935-d5a7-4ab8-b579-247400053b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:47.463249 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34322935-d5a7-4ab8-b579-247400053b20 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:47.463839 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34322935-d5a7-4ab8-b579-247400053b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:47.464306 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34322935-d5a7-4ab8-b579-247400053b20 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:47.465472 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34322935-d5a7-4ab8-b579-247400053b20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:47.469416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34322935-d5a7-4ab8-b579-247400053b20 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:47.475455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37f86d-6c53-4f02-94cf-217ba111f064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79c19d0118be4a2a943f1f8b80c2ee6e, user_id=5bf472768a2c40fba7fef8466253127a, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:47.479610 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37f86d-6c53-4f02-94cf-217ba111f064 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:47.619797 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1387/2771] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:47 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:47.619797 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34322935-d5a7-4ab8-b579-247400053b20 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:47.620895 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-34322935-d5a7-4ab8-b579-247400053b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:47.624319 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1385/2772] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:47 2022] PUT /identity/v3/projects/79c19d0118be4a2a943f1f8b80c2ee6e/users/5bf472768a2c40fba7fef8466253127a/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:47.627956 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37f86d-6c53-4f02-94cf-217ba111f064 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:47.636164 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-6e37f86d-6c53-4f02-94cf-217ba111f064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:47.730764 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37f86d-6c53-4f02-94cf-217ba111f064 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:47.737558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4f475-52c3-4b66-b982-10283d979715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:47.738452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4f475-52c3-4b66-b982-10283d979715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:47.738841 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4f475-52c3-4b66-b982-10283d979715 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:47.739319 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4f475-52c3-4b66-b982-10283d979715 admin admin] PATH_INFO: `/v3/projects/cca5d10d42844de78f582e78089e15f4` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:47.741067 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4f475-52c3-4b66-b982-10283d979715 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cca5d10d42844de78f582e78089e15f4)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:47.744032 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4f475-52c3-4b66-b982-10283d979715 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:47.751190 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34322935-d5a7-4ab8-b579-247400053b20 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:47.819396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:47.829669 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434876e-d54f-48ae-89dc-89e400f3b811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:47.830810 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434876e-d54f-48ae-89dc-89e400f3b811 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:47.831298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434876e-d54f-48ae-89dc-89e400f3b811 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:47.831706 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434876e-d54f-48ae-89dc-89e400f3b811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:47.846135 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434876e-d54f-48ae-89dc-89e400f3b811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:47.847684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d434876e-d54f-48ae-89dc-89e400f3b811 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:47.860454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1388/2773] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:42:47.861284 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434876e-d54f-48ae-89dc-89e400f3b811 service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:47.873447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434876e-d54f-48ae-89dc-89e400f3b811 service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:47.879170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86730da-ab64-4bc4-b11d-266b6c0ac399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:47.879895 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86730da-ab64-4bc4-b11d-266b6c0ac399 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:47.880373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86730da-ab64-4bc4-b11d-266b6c0ac399 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:47.880523 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86730da-ab64-4bc4-b11d-266b6c0ac399 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:47.881165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86730da-ab64-4bc4-b11d-266b6c0ac399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:47.883740 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86730da-ab64-4bc4-b11d-266b6c0ac399 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:47.897105 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1389/2774] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:47.900935 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86730da-ab64-4bc4-b11d-266b6c0ac399 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:47.915685 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86730da-ab64-4bc4-b11d-266b6c0ac399 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:47.916667 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-ada4f475-52c3-4b66-b982-10283d979715 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cca5d10d42844de78f582e78089e15f4', 'request_id': 'req-ada4f475-52c3-4b66-b982-10283d979715'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:47.924362 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-ada4f475-52c3-4b66-b982-10283d979715 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cca5d10d42844de78f582e78089e15f4', 'request_id': 'req-ada4f475-52c3-4b66-b982-10283d979715'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:48.002148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90020744-3cbb-4ad9-bf78-5cfce43541e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:48.003509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90020744-3cbb-4ad9-bf78-5cfce43541e7 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:48.004095 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90020744-3cbb-4ad9-bf78-5cfce43541e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:48.004524 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90020744-3cbb-4ad9-bf78-5cfce43541e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:48.005777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90020744-3cbb-4ad9-bf78-5cfce43541e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:48.009217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90020744-3cbb-4ad9-bf78-5cfce43541e7 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:48.015222 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-ada4f475-52c3-4b66-b982-10283d979715 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cca5d10d42844de78f582e78089e15f4', 'request_id': 'req-ada4f475-52c3-4b66-b982-10283d979715'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:48.027627 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-ada4f475-52c3-4b66-b982-10283d979715 admin admin] The token cache is being invalidate because project cca5d10d42844de78f582e78089e15f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:48.027963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-ada4f475-52c3-4b66-b982-10283d979715 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:48.028606 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-ada4f475-52c3-4b66-b982-10283d979715 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:48.030252 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1390/2775] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:48.031973 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90020744-3cbb-4ad9-bf78-5cfce43541e7 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:48.035067 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-90020744-3cbb-4ad9-bf78-5cfce43541e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:48.036772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1386/2776] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:47 2022] DELETE /identity/v3/projects/cca5d10d42844de78f582e78089e15f4 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:48.037958 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4f475-52c3-4b66-b982-10283d979715 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:48.040297 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ada4f475-52c3-4b66-b982-10283d979715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:48.055027 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4f475-52c3-4b66-b982-10283d979715 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:48.058529 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90020744-3cbb-4ad9-bf78-5cfce43541e7 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:48.060395 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b16c10-9e4b-4d8e-8d58-0693fe917e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:48.061400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b16c10-9e4b-4d8e-8d58-0693fe917e10 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:48.061673 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b16c10-9e4b-4d8e-8d58-0693fe917e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:48.062225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b16c10-9e4b-4d8e-8d58-0693fe917e10 admin admin] PATH_INFO: `/v3/projects/79c19d0118be4a2a943f1f8b80c2ee6e/users/5bf472768a2c40fba7fef8466253127a/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:48.064381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b16c10-9e4b-4d8e-8d58-0693fe917e10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79c19d0118be4a2a943f1f8b80c2ee6e, user_id=5bf472768a2c40fba7fef8466253127a, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:48.067224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b16c10-9e4b-4d8e-8d58-0693fe917e10 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:48.067963 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae852c-22c4-4c37-b6fe-02ffa0e2f219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:48.068905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae852c-22c4-4c37-b6fe-02ffa0e2f219 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:48.069849 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae852c-22c4-4c37-b6fe-02ffa0e2f219 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:48.069849 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae852c-22c4-4c37-b6fe-02ffa0e2f219 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:48.071765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae852c-22c4-4c37-b6fe-02ffa0e2f219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:48.074753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae852c-22c4-4c37-b6fe-02ffa0e2f219 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:48.086635 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1387/2777] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:48 2022] PUT /identity/v3/projects/79c19d0118be4a2a943f1f8b80c2ee6e/users/5bf472768a2c40fba7fef8466253127a/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:48.088512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b16c10-9e4b-4d8e-8d58-0693fe917e10 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:48.091304 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-91b16c10-9e4b-4d8e-8d58-0693fe917e10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:48.095172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1391/2778] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:48.106648 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453437ac-c997-4985-a019-bbaaf2455854 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:48.107172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453437ac-c997-4985-a019-bbaaf2455854 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:48.107491 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453437ac-c997-4985-a019-bbaaf2455854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:48.126603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:48.147423 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-453437ac-c997-4985-a019-bbaaf2455854 None None] MFA Rules not processed for user `5bf472768a2c40fba7fef8466253127a`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:48.149573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-453437ac-c997-4985-a019-bbaaf2455854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:48.208516 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b16c10-9e4b-4d8e-8d58-0693fe917e10 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:48.215628 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa9a0e-4d82-4a64-bbfc-719fc928f1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:48.216561 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa9a0e-4d82-4a64-bbfc-719fc928f1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:48.216970 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa9a0e-4d82-4a64-bbfc-719fc928f1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:48.217317 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa9a0e-4d82-4a64-bbfc-719fc928f1ba admin admin] PATH_INFO: `/v3/users/60ab27cde180418c9c35ab22e707d8eb` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:48.218651 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa9a0e-4d82-4a64-bbfc-719fc928f1ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=60ab27cde180418c9c35ab22e707d8eb)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:48.221005 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa9a0e-4d82-4a64-bbfc-719fc928f1ba admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:48.263514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1392/2779] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:48 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:48.271721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453437ac-c997-4985-a019-bbaaf2455854 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:48.278356 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453437ac-c997-4985-a019-bbaaf2455854 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:48.284877 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de1cfb-f5a1-4bec-a612-84b94f2f8525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:48.285582 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de1cfb-f5a1-4bec-a612-84b94f2f8525 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:48.285874 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de1cfb-f5a1-4bec-a612-84b94f2f8525 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:48.286150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de1cfb-f5a1-4bec-a612-84b94f2f8525 admin admin] PATH_INFO: `/v3/projects/67ec6451da9a4be296d28e10aab80ae7/users/ad8e51b61db044a4824d2eb477e7ab19/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:48.313456 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de1cfb-f5a1-4bec-a612-84b94f2f8525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ec6451da9a4be296d28e10aab80ae7, user_id=ad8e51b61db044a4824d2eb477e7ab19, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:48.319482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de1cfb-f5a1-4bec-a612-84b94f2f8525 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:48.442222 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1393/2780] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:48 2022] PUT /identity/v3/projects/67ec6451da9a4be296d28e10aab80ae7/users/ad8e51b61db044a4824d2eb477e7ab19/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:48.443052 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de1cfb-f5a1-4bec-a612-84b94f2f8525 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:48.445079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f7de1cfb-f5a1-4bec-a612-84b94f2f8525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:48.526379 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:48.529346 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2eaa9a0e-4d82-4a64-bbfc-719fc928f1ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60ab27cde180418c9c35ab22e707d8eb'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:48.553036 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de1cfb-f5a1-4bec-a612-84b94f2f8525 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:48.558550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76ec7d-0a6f-445a-ac36-579b601194f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:48.559259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76ec7d-0a6f-445a-ac36-579b601194f7 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:48.559669 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76ec7d-0a6f-445a-ac36-579b601194f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:48.560020 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76ec7d-0a6f-445a-ac36-579b601194f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:48.627755 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2eaa9a0e-4d82-4a64-bbfc-719fc928f1ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60ab27cde180418c9c35ab22e707d8eb'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:48.628948 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2eaa9a0e-4d82-4a64-bbfc-719fc928f1ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60ab27cde180418c9c35ab22e707d8eb'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:48.630151 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2eaa9a0e-4d82-4a64-bbfc-719fc928f1ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60ab27cde180418c9c35ab22e707d8eb'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:48.640000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-2eaa9a0e-4d82-4a64-bbfc-719fc928f1ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60ab27cde180418c9c35ab22e707d8eb'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:48.668621 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76ec7d-0a6f-445a-ac36-579b601194f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:48.674467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1388/2781] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:48 2022] DELETE /identity/v3/users/60ab27cde180418c9c35ab22e707d8eb => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:48.674765 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76ec7d-0a6f-445a-ac36-579b601194f7 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:48.674869 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66600c-c7a9-4920-8945-577756e769e8 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:48.674869 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66600c-c7a9-4920-8945-577756e769e8 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:48.674869 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66600c-c7a9-4920-8945-577756e769e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:48.687683 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1394/2782] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:48 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:42:48.727772 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:48.748476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-8b66600c-c7a9-4920-8945-577756e769e8 None None] MFA Rules not processed for user `ad8e51b61db044a4824d2eb477e7ab19`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:48.750786 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8b66600c-c7a9-4920-8945-577756e769e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:48.756600 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76ec7d-0a6f-445a-ac36-579b601194f7 service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:48.760350 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-dd76ec7d-0a6f-445a-ac36-579b601194f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:48.782285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76ec7d-0a6f-445a-ac36-579b601194f7 service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:48.852620 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1389/2783] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:48.865661 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66600c-c7a9-4920-8945-577756e769e8 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:48.867727 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8b66600c-c7a9-4920-8945-577756e769e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:48.869746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6fad6-eeb6-418a-8030-fd3e032594e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:48.874644 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6fad6-eeb6-418a-8030-fd3e032594e9 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:48.874644 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6fad6-eeb6-418a-8030-fd3e032594e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:48.874644 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6fad6-eeb6-418a-8030-fd3e032594e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:48.874644 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-4bb6fad6-eeb6-418a-8030-fd3e032594e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:48.883830 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66600c-c7a9-4920-8945-577756e769e8 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:48.889215 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9279977-9ab7-4499-b17f-fc3886c9fddf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:48.890953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9279977-9ab7-4499-b17f-fc3886c9fddf service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:48.890953 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9279977-9ab7-4499-b17f-fc3886c9fddf service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:48.891322 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9279977-9ab7-4499-b17f-fc3886c9fddf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:48.898408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:48.903963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9279977-9ab7-4499-b17f-fc3886c9fddf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:48.905935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9279977-9ab7-4499-b17f-fc3886c9fddf service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:48.919012 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1390/2784] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:42:48.962332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9279977-9ab7-4499-b17f-fc3886c9fddf service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:48.964301 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f9279977-9ab7-4499-b17f-fc3886c9fddf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:49.077248 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6fad6-eeb6-418a-8030-fd3e032594e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:49.081998 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6fad6-eeb6-418a-8030-fd3e032594e9 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:49.102444 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9279977-9ab7-4499-b17f-fc3886c9fddf service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:49.122671 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1395/2785] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:48 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:42:49.124592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105a2f1-645b-48e3-b90e-21a1ba31b169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:49.129070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105a2f1-645b-48e3-b90e-21a1ba31b169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:49.129070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105a2f1-645b-48e3-b90e-21a1ba31b169 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:49.129070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105a2f1-645b-48e3-b90e-21a1ba31b169 admin admin] PATH_INFO: `/v3/users/ad8e51b61db044a4824d2eb477e7ab19` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:49.137398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105a2f1-645b-48e3-b90e-21a1ba31b169 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad8e51b61db044a4824d2eb477e7ab19)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:49.140263 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:49.141534 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105a2f1-645b-48e3-b90e-21a1ba31b169 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:49.324386 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6fad6-eeb6-418a-8030-fd3e032594e9 service cinder] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:49.326548 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-4bb6fad6-eeb6-418a-8030-fd3e032594e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:49.394801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6fad6-eeb6-418a-8030-fd3e032594e9 service cinder] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:49.395570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-f105a2f1-645b-48e3-b90e-21a1ba31b169 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad8e51b61db044a4824d2eb477e7ab19'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:49.408484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9bf11a-37b2-4c8c-b8f5-1cc95903da6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:49.409543 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bf11a-37b2-4c8c-b8f5-1cc95903da6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:49.409990 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bf11a-37b2-4c8c-b8f5-1cc95903da6d admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:49.410470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bf11a-37b2-4c8c-b8f5-1cc95903da6d admin admin] PATH_INFO: `/v3/projects/c60ff568e3094bb899b63b47dfa1bbd1` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:49.412570 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9bf11a-37b2-4c8c-b8f5-1cc95903da6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c60ff568e3094bb899b63b47dfa1bbd1)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:49.414951 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9bf11a-37b2-4c8c-b8f5-1cc95903da6d admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:49.496200 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-f105a2f1-645b-48e3-b90e-21a1ba31b169 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad8e51b61db044a4824d2eb477e7ab19'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:49.497049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-f105a2f1-645b-48e3-b90e-21a1ba31b169 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad8e51b61db044a4824d2eb477e7ab19'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:49.497934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-f105a2f1-645b-48e3-b90e-21a1ba31b169 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad8e51b61db044a4824d2eb477e7ab19'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:49.506666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-f105a2f1-645b-48e3-b90e-21a1ba31b169 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad8e51b61db044a4824d2eb477e7ab19'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:49.525874 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1391/2786] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:48 2022] DELETE /identity/v3/users/ad8e51b61db044a4824d2eb477e7ab19 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:49.530577 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3384e-2ae9-42c2-9cd6-b466276201f6 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:49.531494 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3384e-2ae9-42c2-9cd6-b466276201f6 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:49.531494 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3384e-2ae9-42c2-9cd6-b466276201f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:49.551546 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3f9bf11a-37b2-4c8c-b8f5-1cc95903da6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c60ff568e3094bb899b63b47dfa1bbd1', 'request_id': 'req-3f9bf11a-37b2-4c8c-b8f5-1cc95903da6d'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:49.551821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-08b3384e-2ae9-42c2-9cd6-b466276201f6 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:49.553021 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-08b3384e-2ae9-42c2-9cd6-b466276201f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:49.563615 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3f9bf11a-37b2-4c8c-b8f5-1cc95903da6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c60ff568e3094bb899b63b47dfa1bbd1', 'request_id': 'req-3f9bf11a-37b2-4c8c-b8f5-1cc95903da6d'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:49.639277 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1392/2787] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:42:49.646226 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3384e-2ae9-42c2-9cd6-b466276201f6 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:49.648179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-08b3384e-2ae9-42c2-9cd6-b466276201f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:49.665738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3f9bf11a-37b2-4c8c-b8f5-1cc95903da6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c60ff568e3094bb899b63b47dfa1bbd1', 'request_id': 'req-3f9bf11a-37b2-4c8c-b8f5-1cc95903da6d'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:49.684075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3f9bf11a-37b2-4c8c-b8f5-1cc95903da6d admin admin] The token cache is being invalidate because project c60ff568e3094bb899b63b47dfa1bbd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:49.684606 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3f9bf11a-37b2-4c8c-b8f5-1cc95903da6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:49.685486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-3f9bf11a-37b2-4c8c-b8f5-1cc95903da6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:49.688075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1396/2788] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:49 2022] DELETE /identity/v3/projects/c60ff568e3094bb899b63b47dfa1bbd1 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:49.710077 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569558d-4ecf-43d2-9aaf-f2e1bab628ee None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:49.715490 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569558d-4ecf-43d2-9aaf-f2e1bab628ee None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:49.716197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569558d-4ecf-43d2-9aaf-f2e1bab628ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:49.729769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:49.736069 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3384e-2ae9-42c2-9cd6-b466276201f6 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:49.740982 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf7ad0-a195-46f4-8576-efbff468c7ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:49.742267 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf7ad0-a195-46f4-8576-efbff468c7ca service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:49.742731 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf7ad0-a195-46f4-8576-efbff468c7ca service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:49.743223 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf7ad0-a195-46f4-8576-efbff468c7ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:49.746325 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-9569558d-4ecf-43d2-9aaf-f2e1bab628ee None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:49.746637 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-fedf7ad0-a195-46f4-8576-efbff468c7ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:49.747742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9569558d-4ecf-43d2-9aaf-f2e1bab628ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:49.759084 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1397/2789] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:49.761935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf7ad0-a195-46f4-8576-efbff468c7ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:49.763857 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf7ad0-a195-46f4-8576-efbff468c7ca service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:49.769096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa41931-cb9f-49bb-a790-b076d1c31d8e None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:49.769788 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa41931-cb9f-49bb-a790-b076d1c31d8e None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:49.770419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa41931-cb9f-49bb-a790-b076d1c31d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:49.775781 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1393/2790] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:42:49.805905 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-3fa41931-cb9f-49bb-a790-b076d1c31d8e None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:49.807550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-3fa41931-cb9f-49bb-a790-b076d1c31d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:49.820643 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1398/2791] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:49.833338 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf7ad0-a195-46f4-8576-efbff468c7ca service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:49.852093 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:49.857147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf7ad0-a195-46f4-8576-efbff468c7ca service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:49.864531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5f7c3-bc49-4854-8e26-96da8e8c1b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:49.865173 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5f7c3-bc49-4854-8e26-96da8e8c1b99 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:49.865572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5f7c3-bc49-4854-8e26-96da8e8c1b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:49.865754 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5f7c3-bc49-4854-8e26-96da8e8c1b99 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:49.866476 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b5f7c3-bc49-4854-8e26-96da8e8c1b99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:42:49.866930 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b5f7c3-bc49-4854-8e26-96da8e8c1b99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:49.869488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b5f7c3-bc49-4854-8e26-96da8e8c1b99 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:49.885983 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1394/2792] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:42:49.892664 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa41931-cb9f-49bb-a790-b076d1c31d8e None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:49.915068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa41931-cb9f-49bb-a790-b076d1c31d8e None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:49.923274 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12772251-30cb-4de6-aede-307ad3b5f9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:49.923961 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12772251-30cb-4de6-aede-307ad3b5f9d9 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:49.924384 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12772251-30cb-4de6-aede-307ad3b5f9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:49.925372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12772251-30cb-4de6-aede-307ad3b5f9d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:49.926615 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12772251-30cb-4de6-aede-307ad3b5f9d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:49.929824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12772251-30cb-4de6-aede-307ad3b5f9d9 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:50.089286 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1399/2793] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:49 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:42:50.097163 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5f7c3-bc49-4854-8e26-96da8e8c1b99 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:50.127295 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5f7c3-bc49-4854-8e26-96da8e8c1b99 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:50.291842 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fafd29-667d-4cee-8c1b-072cfd623942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:50.293528 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fafd29-667d-4cee-8c1b-072cfd623942 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:50.294285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fafd29-667d-4cee-8c1b-072cfd623942 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:50.295047 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fafd29-667d-4cee-8c1b-072cfd623942 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:50.295802 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fafd29-667d-4cee-8c1b-072cfd623942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:50.298146 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fafd29-667d-4cee-8c1b-072cfd623942 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:50.407482 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1395/2794] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:50 2022] POST /identity/v3/users => generated 289 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:50.432948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12772251-30cb-4de6-aede-307ad3b5f9d9 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:50.465370 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12772251-30cb-4de6-aede-307ad3b5f9d9 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:50.474314 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0961693-5ebe-4b60-bf2c-07a8f040ede7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:50.475437 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0961693-5ebe-4b60-bf2c-07a8f040ede7 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:50.475908 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0961693-5ebe-4b60-bf2c-07a8f040ede7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:50.477331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0961693-5ebe-4b60-bf2c-07a8f040ede7 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:50.478824 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0961693-5ebe-4b60-bf2c-07a8f040ede7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:50.480861 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0961693-5ebe-4b60-bf2c-07a8f040ede7 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:50.495022 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1400/2795] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:50.499801 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fafd29-667d-4cee-8c1b-072cfd623942 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:50.513496 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fafd29-667d-4cee-8c1b-072cfd623942 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:50.518732 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c898fea-d30f-4ddd-8bb1-e22140031015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:50.519421 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c898fea-d30f-4ddd-8bb1-e22140031015 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:50.519711 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c898fea-d30f-4ddd-8bb1-e22140031015 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:50.520040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c898fea-d30f-4ddd-8bb1-e22140031015 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:50.520820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c898fea-d30f-4ddd-8bb1-e22140031015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:50.522525 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c898fea-d30f-4ddd-8bb1-e22140031015 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:50.543956 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1396/2796] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:42:50.548049 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0961693-5ebe-4b60-bf2c-07a8f040ede7 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:50.559203 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0961693-5ebe-4b60-bf2c-07a8f040ede7 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:50.564505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b0a02-7806-43df-9608-de92fcfa8832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:50.565196 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b0a02-7806-43df-9608-de92fcfa8832 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:50.565468 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b0a02-7806-43df-9608-de92fcfa8832 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:50.565753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b0a02-7806-43df-9608-de92fcfa8832 admin admin] PATH_INFO: `/v3/projects/f5bb7647c9954ad6a54763758dfb117f/users/2eeb13e580d04add94fbea68be7dcb11/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:50.595418 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b0a02-7806-43df-9608-de92fcfa8832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5bb7647c9954ad6a54763758dfb117f, user_id=2eeb13e580d04add94fbea68be7dcb11, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:50.598614 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b0a02-7806-43df-9608-de92fcfa8832 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:50.642559 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c898fea-d30f-4ddd-8bb1-e22140031015 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:50.645600 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-2c898fea-d30f-4ddd-8bb1-e22140031015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:50.670035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c898fea-d30f-4ddd-8bb1-e22140031015 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:50.678557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2db130-c0cd-43d9-a41d-9bf944e3f843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:50.680599 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2db130-c0cd-43d9-a41d-9bf944e3f843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:50.681082 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2db130-c0cd-43d9-a41d-9bf944e3f843 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:50.681567 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2db130-c0cd-43d9-a41d-9bf944e3f843 admin admin] PATH_INFO: `/v3/projects/67ec6451da9a4be296d28e10aab80ae7` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:50.683035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2db130-c0cd-43d9-a41d-9bf944e3f843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67ec6451da9a4be296d28e10aab80ae7)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:50.685182 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2db130-c0cd-43d9-a41d-9bf944e3f843 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:50.692655 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1401/2797] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:50 2022] PUT /identity/v3/projects/f5bb7647c9954ad6a54763758dfb117f/users/2eeb13e580d04add94fbea68be7dcb11/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:50.705733 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e16ab-abd7-4e6d-bc70-de3809218089 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:50.706043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e16ab-abd7-4e6d-bc70-de3809218089 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:50.706367 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e16ab-abd7-4e6d-bc70-de3809218089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:50.764433 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-887e16ab-abd7-4e6d-bc70-de3809218089 None None] MFA Rules not processed for user `2eeb13e580d04add94fbea68be7dcb11`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:50.766682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-887e16ab-abd7-4e6d-bc70-de3809218089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:50.891801 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1402/2798] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:42:50.897230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-bf2db130-c0cd-43d9-a41d-9bf944e3f843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67ec6451da9a4be296d28e10aab80ae7', 'request_id': 'req-bf2db130-c0cd-43d9-a41d-9bf944e3f843'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:50.903094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e16ab-abd7-4e6d-bc70-de3809218089 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:50.907198 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-bf2db130-c0cd-43d9-a41d-9bf944e3f843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67ec6451da9a4be296d28e10aab80ae7', 'request_id': 'req-bf2db130-c0cd-43d9-a41d-9bf944e3f843'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:50.907470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-887e16ab-abd7-4e6d-bc70-de3809218089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:50.988501 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e16ab-abd7-4e6d-bc70-de3809218089 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:51.021738 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-bf2db130-c0cd-43d9-a41d-9bf944e3f843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67ec6451da9a4be296d28e10aab80ae7', 'request_id': 'req-bf2db130-c0cd-43d9-a41d-9bf944e3f843'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:51.037779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-bf2db130-c0cd-43d9-a41d-9bf944e3f843 admin admin] The token cache is being invalidate because project 67ec6451da9a4be296d28e10aab80ae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:51.038341 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-bf2db130-c0cd-43d9-a41d-9bf944e3f843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:51.040014 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-bf2db130-c0cd-43d9-a41d-9bf944e3f843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:51.042585 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1397/2799] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:50 2022] DELETE /identity/v3/projects/67ec6451da9a4be296d28e10aab80ae7 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:51.086213 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744dd82-dcc8-4a85-b2b8-77e7cb3b8667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:51.087151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744dd82-dcc8-4a85-b2b8-77e7cb3b8667 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:51.087724 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744dd82-dcc8-4a85-b2b8-77e7cb3b8667 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:51.088157 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744dd82-dcc8-4a85-b2b8-77e7cb3b8667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:51.091797 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5744dd82-dcc8-4a85-b2b8-77e7cb3b8667 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:51.193314 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744dd82-dcc8-4a85-b2b8-77e7cb3b8667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:51.196818 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744dd82-dcc8-4a85-b2b8-77e7cb3b8667 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:51.211656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1403/2800] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:42:51.411581 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:51.469233 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:55.984094 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2db130-c0cd-43d9-a41d-9bf944e3f843 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:55.986329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-bf2db130-c0cd-43d9-a41d-9bf944e3f843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:56.170645 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2db130-c0cd-43d9-a41d-9bf944e3f843 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:56.182337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fb918-b887-457f-adb5-168dedea2b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:56.182337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fb918-b887-457f-adb5-168dedea2b4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:56.182337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fb918-b887-457f-adb5-168dedea2b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:56.182337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fb918-b887-457f-adb5-168dedea2b4a admin admin] PATH_INFO: `/v3/users/2abc4cab476d47918f16ec53a8106376` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:56.183043 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735fb918-b887-457f-adb5-168dedea2b4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2abc4cab476d47918f16ec53a8106376)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:56.184631 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735fb918-b887-457f-adb5-168dedea2b4a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:56.522406 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-735fb918-b887-457f-adb5-168dedea2b4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2abc4cab476d47918f16ec53a8106376'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:56.667402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-735fb918-b887-457f-adb5-168dedea2b4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2abc4cab476d47918f16ec53a8106376'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:56.671359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-735fb918-b887-457f-adb5-168dedea2b4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2abc4cab476d47918f16ec53a8106376'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:56.672204 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-735fb918-b887-457f-adb5-168dedea2b4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2abc4cab476d47918f16ec53a8106376'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:56.684911 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-735fb918-b887-457f-adb5-168dedea2b4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2abc4cab476d47918f16ec53a8106376'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:56.692303 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:56.715040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1398/2801] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:55 2022] DELETE /identity/v3/users/2abc4cab476d47918f16ec53a8106376 => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:56.718163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9299fed-c525-4eb7-9f30-26cc81736323 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:56.718692 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9299fed-c525-4eb7-9f30-26cc81736323 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:56.719175 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9299fed-c525-4eb7-9f30-26cc81736323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:56.765807 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-a9299fed-c525-4eb7-9f30-26cc81736323 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:56.767997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a9299fed-c525-4eb7-9f30-26cc81736323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:56.926363 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1404/2802] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:42:56.936290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fb918-b887-457f-adb5-168dedea2b4a admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:56.938415 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-735fb918-b887-457f-adb5-168dedea2b4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:57.033881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53b145-c231-49ee-88d8-17403fe61a0c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:57.034319 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53b145-c231-49ee-88d8-17403fe61a0c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:57.034617 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53b145-c231-49ee-88d8-17403fe61a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:57.045226 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fb918-b887-457f-adb5-168dedea2b4a admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:57.051709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc1c4f-f10f-435d-97a3-b0b5f0514b3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:57.052843 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc1c4f-f10f-435d-97a3-b0b5f0514b3e service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:57.053260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc1c4f-f10f-435d-97a3-b0b5f0514b3e service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:57.053715 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc1c4f-f10f-435d-97a3-b0b5f0514b3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:57.068245 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc1c4f-f10f-435d-97a3-b0b5f0514b3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:57.070633 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc1c4f-f10f-435d-97a3-b0b5f0514b3e service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:57.078093 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-fb53b145-c231-49ee-88d8-17403fe61a0c None None] MFA Rules not processed for user `8895cee0f4444866bd4c3908eb248543`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:42:57.079257 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-fb53b145-c231-49ee-88d8-17403fe61a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:57.089565 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1399/2803] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:42:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:42:57.236082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1405/2804] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:42:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:42:57.241202 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc1c4f-f10f-435d-97a3-b0b5f0514b3e service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:57.243451 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c1cc1c4f-f10f-435d-97a3-b0b5f0514b3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:57.258537 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc1c4f-f10f-435d-97a3-b0b5f0514b3e service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:57.272010 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab4111-f42f-47a1-9b8c-f81c0185c81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:57.273474 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab4111-f42f-47a1-9b8c-f81c0185c81e admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:57.273474 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab4111-f42f-47a1-9b8c-f81c0185c81e admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:57.273706 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab4111-f42f-47a1-9b8c-f81c0185c81e admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:57.274406 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab4111-f42f-47a1-9b8c-f81c0185c81e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:57.276477 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab4111-f42f-47a1-9b8c-f81c0185c81e admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:57.420040 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1400/2805] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:42:57 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:57.423408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53b145-c231-49ee-88d8-17403fe61a0c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:57.442188 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53b145-c231-49ee-88d8-17403fe61a0c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:57.465952 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:57.539629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752505e-cbd9-4f2b-92fb-4b8204a74139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:57.540713 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752505e-cbd9-4f2b-92fb-4b8204a74139 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:57.540713 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752505e-cbd9-4f2b-92fb-4b8204a74139 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:57.541351 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752505e-cbd9-4f2b-92fb-4b8204a74139 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:57.541772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752505e-cbd9-4f2b-92fb-4b8204a74139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:57.545440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752505e-cbd9-4f2b-92fb-4b8204a74139 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:57.677695 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1406/2806] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:42:57 2022] POST /identity/v3/users => generated 299 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:42:57.680110 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab4111-f42f-47a1-9b8c-f81c0185c81e admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:57.693985 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab4111-f42f-47a1-9b8c-f81c0185c81e admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:57.698821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d55e5-7e85-4404-b3ba-ea235264f27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:57.699519 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d55e5-7e85-4404-b3ba-ea235264f27e admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:57.699804 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d55e5-7e85-4404-b3ba-ea235264f27e admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:57.700148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d55e5-7e85-4404-b3ba-ea235264f27e admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:57.700805 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9d55e5-7e85-4404-b3ba-ea235264f27e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:57.702532 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9d55e5-7e85-4404-b3ba-ea235264f27e admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:57.715209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1401/2807] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:42:57.719777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752505e-cbd9-4f2b-92fb-4b8204a74139 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:57.732008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752505e-cbd9-4f2b-92fb-4b8204a74139 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:57.737925 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98be75-ab55-437a-ab00-6aace9c6fc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:57.738641 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98be75-ab55-437a-ab00-6aace9c6fc99 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:57.738884 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98be75-ab55-437a-ab00-6aace9c6fc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:57.739281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98be75-ab55-437a-ab00-6aace9c6fc99 admin admin] PATH_INFO: `/v3/projects/e1c30057405b424b8ed90e2a9e518f12/users/50c2f9a37b674adc85e943ee3bf60b50/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:57.759260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98be75-ab55-437a-ab00-6aace9c6fc99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c30057405b424b8ed90e2a9e518f12, user_id=50c2f9a37b674adc85e943ee3bf60b50, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:57.761085 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d55e5-7e85-4404-b3ba-ea235264f27e admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:57.762157 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98be75-ab55-437a-ab00-6aace9c6fc99 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:57.763293 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-1c9d55e5-7e85-4404-b3ba-ea235264f27e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:57.802643 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d55e5-7e85-4404-b3ba-ea235264f27e admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:57.810363 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7460907-53c4-4096-adf5-2cef36b3396d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:57.811399 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7460907-53c4-4096-adf5-2cef36b3396d admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:57.811904 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7460907-53c4-4096-adf5-2cef36b3396d admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:57.813194 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7460907-53c4-4096-adf5-2cef36b3396d admin admin] PATH_INFO: `/v3/projects/cc1f99ce86c4420ca9afc560cc5a3155` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:57.814050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7460907-53c4-4096-adf5-2cef36b3396d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc1f99ce86c4420ca9afc560cc5a3155)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:57.817447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7460907-53c4-4096-adf5-2cef36b3396d admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:57.883986 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1407/2808] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:57 2022] PUT /identity/v3/projects/e1c30057405b424b8ed90e2a9e518f12/users/50c2f9a37b674adc85e943ee3bf60b50/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:57.889433 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98be75-ab55-437a-ab00-6aace9c6fc99 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:57.962717 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98be75-ab55-437a-ab00-6aace9c6fc99 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:57.973839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b7460907-53c4-4096-adf5-2cef36b3396d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc1f99ce86c4420ca9afc560cc5a3155', 'request_id': 'req-b7460907-53c4-4096-adf5-2cef36b3396d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:57.983464 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b7460907-53c4-4096-adf5-2cef36b3396d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc1f99ce86c4420ca9afc560cc5a3155', 'request_id': 'req-b7460907-53c4-4096-adf5-2cef36b3396d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:58.051866 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae56a38-6710-4d6f-8fcc-cb445911d7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:58.053133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae56a38-6710-4d6f-8fcc-cb445911d7ae admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:58.053688 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae56a38-6710-4d6f-8fcc-cb445911d7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:58.055021 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae56a38-6710-4d6f-8fcc-cb445911d7ae admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:58.056116 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae56a38-6710-4d6f-8fcc-cb445911d7ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:58.059365 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae56a38-6710-4d6f-8fcc-cb445911d7ae admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:58.062327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b7460907-53c4-4096-adf5-2cef36b3396d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc1f99ce86c4420ca9afc560cc5a3155', 'request_id': 'req-b7460907-53c4-4096-adf5-2cef36b3396d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:58.080212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1408/2809] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:42:58.083203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b7460907-53c4-4096-adf5-2cef36b3396d admin admin] The token cache is being invalidate because project cc1f99ce86c4420ca9afc560cc5a3155 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:42:58.083659 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b7460907-53c4-4096-adf5-2cef36b3396d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:58.084640 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae56a38-6710-4d6f-8fcc-cb445911d7ae admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:58.085572 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b7460907-53c4-4096-adf5-2cef36b3396d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:58.088128 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1402/2810] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:57 2022] DELETE /identity/v3/projects/cc1f99ce86c4420ca9afc560cc5a3155 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:58.092674 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7460907-53c4-4096-adf5-2cef36b3396d admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:58.095209 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b7460907-53c4-4096-adf5-2cef36b3396d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:58.106167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae56a38-6710-4d6f-8fcc-cb445911d7ae admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:58.113674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2039e-923d-47a7-87ee-258e3f42b0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:58.115007 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2039e-923d-47a7-87ee-258e3f42b0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:58.115680 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2039e-923d-47a7-87ee-258e3f42b0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:58.116219 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f2039e-923d-47a7-87ee-258e3f42b0f9 admin admin] PATH_INFO: `/v3/projects/e1c30057405b424b8ed90e2a9e518f12/users/50c2f9a37b674adc85e943ee3bf60b50/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:58.119558 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7460907-53c4-4096-adf5-2cef36b3396d admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:58.121357 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f2039e-923d-47a7-87ee-258e3f42b0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c30057405b424b8ed90e2a9e518f12, user_id=50c2f9a37b674adc85e943ee3bf60b50, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:58.124613 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f2039e-923d-47a7-87ee-258e3f42b0f9 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:58.127381 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f55237-2f4e-45b4-b7b2-1f64b2420ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:58.128616 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f55237-2f4e-45b4-b7b2-1f64b2420ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:58.133692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f55237-2f4e-45b4-b7b2-1f64b2420ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:58.133692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f55237-2f4e-45b4-b7b2-1f64b2420ef9 admin admin] PATH_INFO: `/v3/users/5bf472768a2c40fba7fef8466253127a` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:58.133692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f55237-2f4e-45b4-b7b2-1f64b2420ef9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bf472768a2c40fba7fef8466253127a)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:58.134291 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f55237-2f4e-45b4-b7b2-1f64b2420ef9 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:58.139225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1409/2811] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:42:58 2022] PUT /identity/v3/projects/e1c30057405b424b8ed90e2a9e518f12/users/50c2f9a37b674adc85e943ee3bf60b50/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:42:58.543907 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:58.564927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-73f55237-2f4e-45b4-b7b2-1f64b2420ef9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bf472768a2c40fba7fef8466253127a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:58.670664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-73f55237-2f4e-45b4-b7b2-1f64b2420ef9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bf472768a2c40fba7fef8466253127a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:58.670664 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-73f55237-2f4e-45b4-b7b2-1f64b2420ef9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bf472768a2c40fba7fef8466253127a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:58.671179 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-73f55237-2f4e-45b4-b7b2-1f64b2420ef9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bf472768a2c40fba7fef8466253127a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:58.685828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-73f55237-2f4e-45b4-b7b2-1f64b2420ef9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bf472768a2c40fba7fef8466253127a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:58.744759 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1403/2812] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:42:58 2022] DELETE /identity/v3/users/5bf472768a2c40fba7fef8466253127a => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:42:59.404785 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2039e-923d-47a7-87ee-258e3f42b0f9 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:42:59.407846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-d5f2039e-923d-47a7-87ee-258e3f42b0f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:42:59.442846 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:42:59.568506 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f2039e-923d-47a7-87ee-258e3f42b0f9 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:42:59.670879 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d43cc9-a274-47d2-ae1e-87114e42354b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:42:59.671598 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d43cc9-a274-47d2-ae1e-87114e42354b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:42:59.671881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d43cc9-a274-47d2-ae1e-87114e42354b admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:42:59.672153 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d43cc9-a274-47d2-ae1e-87114e42354b admin admin] PATH_INFO: `/v3/projects/79c19d0118be4a2a943f1f8b80c2ee6e` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:42:59.673697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d43cc9-a274-47d2-ae1e-87114e42354b admin admin] RBAC: Authorizing `identity:delete_project(project_id=79c19d0118be4a2a943f1f8b80c2ee6e)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:42:59.675549 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d43cc9-a274-47d2-ae1e-87114e42354b admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:42:59.874350 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e2d43cc9-a274-47d2-ae1e-87114e42354b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79c19d0118be4a2a943f1f8b80c2ee6e', 'request_id': 'req-e2d43cc9-a274-47d2-ae1e-87114e42354b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:42:59.887631 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e2d43cc9-a274-47d2-ae1e-87114e42354b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79c19d0118be4a2a943f1f8b80c2ee6e', 'request_id': 'req-e2d43cc9-a274-47d2-ae1e-87114e42354b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:00.068303 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e2d43cc9-a274-47d2-ae1e-87114e42354b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79c19d0118be4a2a943f1f8b80c2ee6e', 'request_id': 'req-e2d43cc9-a274-47d2-ae1e-87114e42354b'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:00.090758 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e2d43cc9-a274-47d2-ae1e-87114e42354b admin admin] The token cache is being invalidate because project 79c19d0118be4a2a943f1f8b80c2ee6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:43:00.091271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e2d43cc9-a274-47d2-ae1e-87114e42354b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:00.093122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e2d43cc9-a274-47d2-ae1e-87114e42354b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:00.094339 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1410/2813] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:42:59 2022] DELETE /identity/v3/projects/79c19d0118be4a2a943f1f8b80c2ee6e => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:43:00.100829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d763478-83f3-4ee4-a685-3d23301376be None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:00.102641 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d763478-83f3-4ee4-a685-3d23301376be None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:00.103166 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d763478-83f3-4ee4-a685-3d23301376be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:00.131169 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-9d763478-83f3-4ee4-a685-3d23301376be None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:00.134471 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-9d763478-83f3-4ee4-a685-3d23301376be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:00.274573 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1404/2814] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:43:00.288962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0349d-7585-4fa5-9d84-4eea1b31ab0c None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:00.289557 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0349d-7585-4fa5-9d84-4eea1b31ab0c None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:00.290075 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0349d-7585-4fa5-9d84-4eea1b31ab0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:00.327724 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-b6e0349d-7585-4fa5-9d84-4eea1b31ab0c None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:00.330114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b6e0349d-7585-4fa5-9d84-4eea1b31ab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:00.346942 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1411/2815] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:43:00.353192 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d763478-83f3-4ee4-a685-3d23301376be None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:00.377273 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d763478-83f3-4ee4-a685-3d23301376be None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:00.390001 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a815a-95e8-4ac9-bb3f-f3c542c19048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:00.391884 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a815a-95e8-4ac9-bb3f-f3c542c19048 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:00.392677 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a815a-95e8-4ac9-bb3f-f3c542c19048 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:00.393142 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a815a-95e8-4ac9-bb3f-f3c542c19048 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:00.394350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a815a-95e8-4ac9-bb3f-f3c542c19048 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:43:00.395230 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a815a-95e8-4ac9-bb3f-f3c542c19048 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:00.399317 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a815a-95e8-4ac9-bb3f-f3c542c19048 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:00.424082 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1405/2816] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:43:00.429762 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0349d-7585-4fa5-9d84-4eea1b31ab0c None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:00.441862 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0349d-7585-4fa5-9d84-4eea1b31ab0c None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:00.449392 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30591bd1-deba-4b49-b1e0-b24bebd18fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:00.458439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30591bd1-deba-4b49-b1e0-b24bebd18fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:00.458439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30591bd1-deba-4b49-b1e0-b24bebd18fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:00.458439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30591bd1-deba-4b49-b1e0-b24bebd18fc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:00.458439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30591bd1-deba-4b49-b1e0-b24bebd18fc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:00.458439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30591bd1-deba-4b49-b1e0-b24bebd18fc4 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:00.602198 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1412/2817] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:43:00 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:43:00.608867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a815a-95e8-4ac9-bb3f-f3c542c19048 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:00.632723 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a815a-95e8-4ac9-bb3f-f3c542c19048 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:00.736835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a4809-d0b3-4057-a5ae-39b2f7fc3d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:00.737566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a4809-d0b3-4057-a5ae-39b2f7fc3d6b admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:00.737901 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a4809-d0b3-4057-a5ae-39b2f7fc3d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:00.738234 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a4809-d0b3-4057-a5ae-39b2f7fc3d6b admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:00.739029 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a4809-d0b3-4057-a5ae-39b2f7fc3d6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:00.742692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a4809-d0b3-4057-a5ae-39b2f7fc3d6b admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:00.889455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1406/2818] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:43:00 2022] POST /identity/v3/users => generated 292 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:43:00.893412 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30591bd1-deba-4b49-b1e0-b24bebd18fc4 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:00.917564 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30591bd1-deba-4b49-b1e0-b24bebd18fc4 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:00.930937 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60215b0-86b6-4b7b-a6c2-cfe0fd8e69d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:00.939170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60215b0-86b6-4b7b-a6c2-cfe0fd8e69d8 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:00.939170 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60215b0-86b6-4b7b-a6c2-cfe0fd8e69d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:00.939608 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60215b0-86b6-4b7b-a6c2-cfe0fd8e69d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:00.940464 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60215b0-86b6-4b7b-a6c2-cfe0fd8e69d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:00.942648 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60215b0-86b6-4b7b-a6c2-cfe0fd8e69d8 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:00.967923 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1413/2819] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:43:00.974608 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a4809-d0b3-4057-a5ae-39b2f7fc3d6b admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:00.995057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a4809-d0b3-4057-a5ae-39b2f7fc3d6b admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:01.005076 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d52fb-cfa6-4232-a8dd-3cd1a68d8008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:01.006306 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d52fb-cfa6-4232-a8dd-3cd1a68d8008 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:01.007565 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d52fb-cfa6-4232-a8dd-3cd1a68d8008 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:01.008282 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d52fb-cfa6-4232-a8dd-3cd1a68d8008 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:01.009949 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d52fb-cfa6-4232-a8dd-3cd1a68d8008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:01.012234 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d52fb-cfa6-4232-a8dd-3cd1a68d8008 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:01.031533 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1407/2820] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:43:01.036897 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60215b0-86b6-4b7b-a6c2-cfe0fd8e69d8 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:01.073555 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60215b0-86b6-4b7b-a6c2-cfe0fd8e69d8 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:01.087881 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942dbb22-4b5f-4ac4-8e2f-af3600e6ea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:01.089505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dbb22-4b5f-4ac4-8e2f-af3600e6ea8f admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:01.090103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dbb22-4b5f-4ac4-8e2f-af3600e6ea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:01.090690 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dbb22-4b5f-4ac4-8e2f-af3600e6ea8f admin admin] PATH_INFO: `/v3/projects/e6f75c0f90bc401d822f36ded5005764/users/3e289991b50b410abb2ecda49668de75/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:01.118629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:01.124983 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942dbb22-4b5f-4ac4-8e2f-af3600e6ea8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6f75c0f90bc401d822f36ded5005764, user_id=3e289991b50b410abb2ecda49668de75, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:01.131571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942dbb22-4b5f-4ac4-8e2f-af3600e6ea8f admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:01.306417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1414/2821] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:43:01 2022] PUT /identity/v3/projects/e6f75c0f90bc401d822f36ded5005764/users/3e289991b50b410abb2ecda49668de75/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:43:01.312674 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ddb52-f171-46b5-8a2e-ad0d17934b6e None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:01.312674 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ddb52-f171-46b5-8a2e-ad0d17934b6e None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:01.312674 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ddb52-f171-46b5-8a2e-ad0d17934b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:01.357468 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-e36ddb52-f171-46b5-8a2e-ad0d17934b6e None None] MFA Rules not processed for user `3e289991b50b410abb2ecda49668de75`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:01.360327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e36ddb52-f171-46b5-8a2e-ad0d17934b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:01.372916 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:01.472560 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1408/2822] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:43:01.484482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942dbb22-4b5f-4ac4-8e2f-af3600e6ea8f admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:01.626209 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942dbb22-4b5f-4ac4-8e2f-af3600e6ea8f admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:01.638488 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6795db-e257-4fa4-b35c-8f6c3b91a965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:01.639289 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6795db-e257-4fa4-b35c-8f6c3b91a965 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:01.639969 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6795db-e257-4fa4-b35c-8f6c3b91a965 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:01.640296 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6795db-e257-4fa4-b35c-8f6c3b91a965 admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:01.641126 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6795db-e257-4fa4-b35c-8f6c3b91a965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:01.643259 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6795db-e257-4fa4-b35c-8f6c3b91a965 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:01.772255 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1415/2823] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:43:01 2022] POST /identity/v3/projects => generated 383 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:43:01.778729 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ddb52-f171-46b5-8a2e-ad0d17934b6e None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:01.807980 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ddb52-f171-46b5-8a2e-ad0d17934b6e None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:01.951482 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3742d7f-b3e4-4695-861a-b9b3ec0fb889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:01.952771 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3742d7f-b3e4-4695-861a-b9b3ec0fb889 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:01.953448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3742d7f-b3e4-4695-861a-b9b3ec0fb889 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:01.953981 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3742d7f-b3e4-4695-861a-b9b3ec0fb889 admin admin] PATH_INFO: `/v3/users` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:01.955296 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3742d7f-b3e4-4695-861a-b9b3ec0fb889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:01.958561 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3742d7f-b3e4-4695-861a-b9b3ec0fb889 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:02.142692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1409/2824] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:43:01 2022] POST /identity/v3/users => generated 298 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:43:02.144734 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6795db-e257-4fa4-b35c-8f6c3b91a965 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:02.169600 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6795db-e257-4fa4-b35c-8f6c3b91a965 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:02.186758 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d193a-da23-44df-93e6-392cc8ed59b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:02.188351 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d193a-da23-44df-93e6-392cc8ed59b2 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:02.188741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d193a-da23-44df-93e6-392cc8ed59b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:02.189243 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d193a-da23-44df-93e6-392cc8ed59b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:02.190402 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d193a-da23-44df-93e6-392cc8ed59b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:02.194062 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d193a-da23-44df-93e6-392cc8ed59b2 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:02.248285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1416/2825] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:43:02.250064 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3742d7f-b3e4-4695-861a-b9b3ec0fb889 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:02.284174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3742d7f-b3e4-4695-861a-b9b3ec0fb889 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:02.309447 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fc4e9-d817-4a4e-af49-6a606fa59599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:02.310296 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fc4e9-d817-4a4e-af49-6a606fa59599 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:02.310794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fc4e9-d817-4a4e-af49-6a606fa59599 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:02.311326 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fc4e9-d817-4a4e-af49-6a606fa59599 admin admin] PATH_INFO: `/v3/projects/783d28d2c0d14214a2c758945cdb22b4/users/3c3354e5321c443cb5a60bd78d6ef198/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:02.360737 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7fc4e9-d817-4a4e-af49-6a606fa59599 admin admin] RBAC: Authorizing `identity:create_grant(project_id=783d28d2c0d14214a2c758945cdb22b4, user_id=3c3354e5321c443cb5a60bd78d6ef198, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:02.364722 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7fc4e9-d817-4a4e-af49-6a606fa59599 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:02.452197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1410/2826] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:43:02 2022] PUT /identity/v3/projects/783d28d2c0d14214a2c758945cdb22b4/users/3c3354e5321c443cb5a60bd78d6ef198/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:43:02.456658 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d193a-da23-44df-93e6-392cc8ed59b2 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:02.490051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d193a-da23-44df-93e6-392cc8ed59b2 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:02.708137 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89cfa0-29cb-43ed-bbac-cb8fd9c6f4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:02.709571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89cfa0-29cb-43ed-bbac-cb8fd9c6f4dd admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:02.710796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89cfa0-29cb-43ed-bbac-cb8fd9c6f4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:02.711521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89cfa0-29cb-43ed-bbac-cb8fd9c6f4dd admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:02.718672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89cfa0-29cb-43ed-bbac-cb8fd9c6f4dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:02.722067 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89cfa0-29cb-43ed-bbac-cb8fd9c6f4dd admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:02.755037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1417/2827] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:43:02.757708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fc4e9-d817-4a4e-af49-6a606fa59599 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:02.771756 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fc4e9-d817-4a4e-af49-6a606fa59599 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:02.778527 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c01743-723c-443a-8a24-a12a01afc0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:02.779257 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c01743-723c-443a-8a24-a12a01afc0cb admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:02.779612 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c01743-723c-443a-8a24-a12a01afc0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:02.779974 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c01743-723c-443a-8a24-a12a01afc0cb admin admin] PATH_INFO: `/v3/projects/783d28d2c0d14214a2c758945cdb22b4/users/3c3354e5321c443cb5a60bd78d6ef198/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:02.782704 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c01743-723c-443a-8a24-a12a01afc0cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=783d28d2c0d14214a2c758945cdb22b4, user_id=3c3354e5321c443cb5a60bd78d6ef198, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:02.785936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c01743-723c-443a-8a24-a12a01afc0cb admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:02.799392 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1411/2828] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:43:02 2022] PUT /identity/v3/projects/783d28d2c0d14214a2c758945cdb22b4/users/3c3354e5321c443cb5a60bd78d6ef198/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:43:02.814492 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407cc33-f360-4752-ba10-4282db828607 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:02.814783 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407cc33-f360-4752-ba10-4282db828607 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:02.814927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407cc33-f360-4752-ba10-4282db828607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:02.862461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-7407cc33-f360-4752-ba10-4282db828607 None None] MFA Rules not processed for user `3c3354e5321c443cb5a60bd78d6ef198`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:02.863280 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-7407cc33-f360-4752-ba10-4282db828607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:02.966418 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:02.995163 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1418/2829] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:43:03.010307 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c01743-723c-443a-8a24-a12a01afc0cb admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:03.012324 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e6c01743-723c-443a-8a24-a12a01afc0cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:03.117397 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c01743-723c-443a-8a24-a12a01afc0cb admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:03.127117 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e4006-8790-42b6-b079-b358bb74730e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:03.128557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e4006-8790-42b6-b079-b358bb74730e service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:03.130590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e4006-8790-42b6-b079-b358bb74730e service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:03.131259 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e4006-8790-42b6-b079-b358bb74730e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:03.167459 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e4006-8790-42b6-b079-b358bb74730e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:03.170310 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e4006-8790-42b6-b079-b358bb74730e service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:03.193838 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1412/2830] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:43:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:43:03.411579 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407cc33-f360-4752-ba10-4282db828607 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:03.416010 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-7407cc33-f360-4752-ba10-4282db828607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:03.461331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407cc33-f360-4752-ba10-4282db828607 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:03.474396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216f4aa-d235-4881-9b33-005398bcda51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:03.475616 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216f4aa-d235-4881-9b33-005398bcda51 service cinder] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:03.483604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216f4aa-d235-4881-9b33-005398bcda51 service cinder] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:03.483604 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216f4aa-d235-4881-9b33-005398bcda51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:03.626411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216f4aa-d235-4881-9b33-005398bcda51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:03.631209 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216f4aa-d235-4881-9b33-005398bcda51 service cinder] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:03.656376 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1419/2831] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:43:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:43:04.540015 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e4006-8790-42b6-b079-b358bb74730e service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:04.545034 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:04.682530 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e4006-8790-42b6-b079-b358bb74730e service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:04.688899 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82593d-96cb-43df-9285-2303c128c4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:04.690701 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82593d-96cb-43df-9285-2303c128c4fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:04.691124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82593d-96cb-43df-9285-2303c128c4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:04.691469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82593d-96cb-43df-9285-2303c128c4fc admin admin] PATH_INFO: `/v3/users/3e289991b50b410abb2ecda49668de75` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:04.693145 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82593d-96cb-43df-9285-2303c128c4fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e289991b50b410abb2ecda49668de75)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:04.697555 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82593d-96cb-43df-9285-2303c128c4fc admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:04.993662 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-4b82593d-96cb-43df-9285-2303c128c4fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e289991b50b410abb2ecda49668de75'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:05.109445 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-4b82593d-96cb-43df-9285-2303c128c4fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e289991b50b410abb2ecda49668de75'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:05.110633 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-4b82593d-96cb-43df-9285-2303c128c4fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e289991b50b410abb2ecda49668de75'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:05.111881 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-4b82593d-96cb-43df-9285-2303c128c4fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e289991b50b410abb2ecda49668de75'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:05.130319 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-4b82593d-96cb-43df-9285-2303c128c4fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e289991b50b410abb2ecda49668de75'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:05.155035 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1413/2832] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:43:04 2022] DELETE /identity/v3/users/3e289991b50b410abb2ecda49668de75 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:43:05.160408 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924d63c-ea1d-458e-92f5-da75e1827252 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:05.161282 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924d63c-ea1d-458e-92f5-da75e1827252 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:05.161282 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924d63c-ea1d-458e-92f5-da75e1827252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:05.200058 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-5924d63c-ea1d-458e-92f5-da75e1827252 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:05.204861 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5924d63c-ea1d-458e-92f5-da75e1827252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:05.408055 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1420/2833] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:43:05.419224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82593d-96cb-43df-9285-2303c128c4fc admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:05.421638 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-4b82593d-96cb-43df-9285-2303c128c4fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:05.520757 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82593d-96cb-43df-9285-2303c128c4fc admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:05.526091 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03485c-2428-4ac4-8935-af355d4734be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:05.527038 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03485c-2428-4ac4-8935-af355d4734be service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:05.527417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03485c-2428-4ac4-8935-af355d4734be service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:05.527829 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03485c-2428-4ac4-8935-af355d4734be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:05.541490 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03485c-2428-4ac4-8935-af355d4734be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:05.544147 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03485c-2428-4ac4-8935-af355d4734be service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:05.563565 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1414/2834] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:43:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:43:06.189104 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924d63c-ea1d-458e-92f5-da75e1827252 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:06.200476 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-5924d63c-ea1d-458e-92f5-da75e1827252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:06.239996 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924d63c-ea1d-458e-92f5-da75e1827252 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:06.249293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e50f2a-52bd-430d-94b7-42a993914623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:06.250491 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e50f2a-52bd-430d-94b7-42a993914623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:06.251145 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e50f2a-52bd-430d-94b7-42a993914623 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:06.252396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e50f2a-52bd-430d-94b7-42a993914623 admin admin] PATH_INFO: `/v3/projects/e6f75c0f90bc401d822f36ded5005764` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:06.254411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e50f2a-52bd-430d-94b7-42a993914623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6f75c0f90bc401d822f36ded5005764)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:06.258721 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e50f2a-52bd-430d-94b7-42a993914623 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:06.520440 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-b9e50f2a-52bd-430d-94b7-42a993914623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6f75c0f90bc401d822f36ded5005764', 'request_id': 'req-b9e50f2a-52bd-430d-94b7-42a993914623'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:06.543960 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-b9e50f2a-52bd-430d-94b7-42a993914623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6f75c0f90bc401d822f36ded5005764', 'request_id': 'req-b9e50f2a-52bd-430d-94b7-42a993914623'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:06.639584 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-b9e50f2a-52bd-430d-94b7-42a993914623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6f75c0f90bc401d822f36ded5005764', 'request_id': 'req-b9e50f2a-52bd-430d-94b7-42a993914623'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:06.677515 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-b9e50f2a-52bd-430d-94b7-42a993914623 admin admin] The token cache is being invalidate because project e6f75c0f90bc401d822f36ded5005764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:43:06.678051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-b9e50f2a-52bd-430d-94b7-42a993914623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:06.680719 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-b9e50f2a-52bd-430d-94b7-42a993914623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:06.684380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1421/2835] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:43:06 2022] DELETE /identity/v3/projects/e6f75c0f90bc401d822f36ded5005764 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:43:06.690254 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03485c-2428-4ac4-8935-af355d4734be service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:06.711773 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03485c-2428-4ac4-8935-af355d4734be service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:06.762934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:06.843249 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ded7f2-51e2-4bf6-8540-3141514f0084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:06.844401 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ded7f2-51e2-4bf6-8540-3141514f0084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:06.845020 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ded7f2-51e2-4bf6-8540-3141514f0084 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:06.845531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ded7f2-51e2-4bf6-8540-3141514f0084 admin admin] PATH_INFO: `/v3/users/3c3354e5321c443cb5a60bd78d6ef198` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:06.847173 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ded7f2-51e2-4bf6-8540-3141514f0084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c3354e5321c443cb5a60bd78d6ef198)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:06.849495 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ded7f2-51e2-4bf6-8540-3141514f0084 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:07.207826 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-35ded7f2-51e2-4bf6-8540-3141514f0084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c3354e5321c443cb5a60bd78d6ef198'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:07.308364 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-35ded7f2-51e2-4bf6-8540-3141514f0084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c3354e5321c443cb5a60bd78d6ef198'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:07.309689 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-35ded7f2-51e2-4bf6-8540-3141514f0084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c3354e5321c443cb5a60bd78d6ef198'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:07.310839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-35ded7f2-51e2-4bf6-8540-3141514f0084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c3354e5321c443cb5a60bd78d6ef198'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:07.324912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-35ded7f2-51e2-4bf6-8540-3141514f0084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c3354e5321c443cb5a60bd78d6ef198'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:07.354482 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1415/2836] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:43:06 2022] DELETE /identity/v3/users/3c3354e5321c443cb5a60bd78d6ef198 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:43:08.146760 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e50f2a-52bd-430d-94b7-42a993914623 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:08.151607 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-b9e50f2a-52bd-430d-94b7-42a993914623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:08.176006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e50f2a-52bd-430d-94b7-42a993914623 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:08.183821 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:08.273033 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bd2c4-6711-4ae3-b56b-5828df68723c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:08.273550 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bd2c4-6711-4ae3-b56b-5828df68723c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:08.273787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bd2c4-6711-4ae3-b56b-5828df68723c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:08.286681 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f5d1a-5a3a-4293-8e6e-be6f29880333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:08.287582 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f5d1a-5a3a-4293-8e6e-be6f29880333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:08.288583 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f5d1a-5a3a-4293-8e6e-be6f29880333 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:08.288962 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f5d1a-5a3a-4293-8e6e-be6f29880333 admin admin] PATH_INFO: `/v3/projects/783d28d2c0d14214a2c758945cdb22b4` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:08.291014 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f5d1a-5a3a-4293-8e6e-be6f29880333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=783d28d2c0d14214a2c758945cdb22b4)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:08.294432 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f5d1a-5a3a-4293-8e6e-be6f29880333 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:08.324305 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b32bd2c4-6711-4ae3-b56b-5828df68723c None None] MFA Rules not processed for user `50c2f9a37b674adc85e943ee3bf60b50`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:08.325925 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b32bd2c4-6711-4ae3-b56b-5828df68723c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:08.411422 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1416/2837] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:43:08.425452 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bd2c4-6711-4ae3-b56b-5828df68723c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:08.427423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b32bd2c4-6711-4ae3-b56b-5828df68723c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:08.463666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:08.493312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bd2c4-6711-4ae3-b56b-5828df68723c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:08.497571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aff77f-9919-479d-b35c-ce53e6fbf5e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1a48805d95640489d7ce3333dd687b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:08.498192 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aff77f-9919-479d-b35c-ce53e6fbf5e1 service glance] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:08.504618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aff77f-9919-479d-b35c-ce53e6fbf5e1 service glance] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:08.504618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aff77f-9919-479d-b35c-ce53e6fbf5e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:08.504618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-97aff77f-9919-479d-b35c-ce53e6fbf5e1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:08.558068 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-fe8f5d1a-5a3a-4293-8e6e-be6f29880333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '783d28d2c0d14214a2c758945cdb22b4', 'request_id': 'req-fe8f5d1a-5a3a-4293-8e6e-be6f29880333'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:08.566215 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-fe8f5d1a-5a3a-4293-8e6e-be6f29880333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '783d28d2c0d14214a2c758945cdb22b4', 'request_id': 'req-fe8f5d1a-5a3a-4293-8e6e-be6f29880333'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:08.586416 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aff77f-9919-479d-b35c-ce53e6fbf5e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:08.588839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aff77f-9919-479d-b35c-ce53e6fbf5e1 service glance] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:08.608858 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1417/2838] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:43:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:43:08.696946 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-fe8f5d1a-5a3a-4293-8e6e-be6f29880333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '783d28d2c0d14214a2c758945cdb22b4', 'request_id': 'req-fe8f5d1a-5a3a-4293-8e6e-be6f29880333'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:08.712664 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-fe8f5d1a-5a3a-4293-8e6e-be6f29880333 admin admin] The token cache is being invalidate because project 783d28d2c0d14214a2c758945cdb22b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:43:08.713090 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-fe8f5d1a-5a3a-4293-8e6e-be6f29880333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:08.713803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-fe8f5d1a-5a3a-4293-8e6e-be6f29880333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:08.715554 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1422/2839] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:43:08 2022] DELETE /identity/v3/projects/783d28d2c0d14214a2c758945cdb22b4 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:43:08.722438 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cfb1f-0043-4a2a-b22a-d6576ed8a2bd None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:08.722965 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cfb1f-0043-4a2a-b22a-d6576ed8a2bd None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:08.723515 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cfb1f-0043-4a2a-b22a-d6576ed8a2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:08.752670 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-973cfb1f-0043-4a2a-b22a-d6576ed8a2bd None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:08.754252 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-973cfb1f-0043-4a2a-b22a-d6576ed8a2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:08.859104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1418/2840] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:43:08.867373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8414062-eafa-443b-a3f1-863f04a3d27f None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:08.867666 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8414062-eafa-443b-a3f1-863f04a3d27f None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:08.867988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8414062-eafa-443b-a3f1-863f04a3d27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:08.892718 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-e8414062-eafa-443b-a3f1-863f04a3d27f None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:08.894433 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e8414062-eafa-443b-a3f1-863f04a3d27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:08.905473 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1423/2841] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:43:08.911977 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cfb1f-0043-4a2a-b22a-d6576ed8a2bd None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:08.923828 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cfb1f-0043-4a2a-b22a-d6576ed8a2bd None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:08.929396 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a957d4-76ad-4379-8c8d-80755909a9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:08.930194 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a957d4-76ad-4379-8c8d-80755909a9c9 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:08.930860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a957d4-76ad-4379-8c8d-80755909a9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:08.931281 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a957d4-76ad-4379-8c8d-80755909a9c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:08.932018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a957d4-76ad-4379-8c8d-80755909a9c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:43:08.932409 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a957d4-76ad-4379-8c8d-80755909a9c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:08.934508 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a957d4-76ad-4379-8c8d-80755909a9c9 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:08.951010 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1419/2842] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:43:08.953026 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8414062-eafa-443b-a3f1-863f04a3d27f None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:08.964417 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8414062-eafa-443b-a3f1-863f04a3d27f None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:08.969580 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d87a8-e5d4-4f56-9590-a84bc267b65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:08.970419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d87a8-e5d4-4f56-9590-a84bc267b65b admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:08.970833 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d87a8-e5d4-4f56-9590-a84bc267b65b admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:08.971231 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d87a8-e5d4-4f56-9590-a84bc267b65b admin admin] PATH_INFO: `/v3/projects` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:08.972035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d87a8-e5d4-4f56-9590-a84bc267b65b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:08.974865 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d87a8-e5d4-4f56-9590-a84bc267b65b admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:09.005630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962976d-df37-4ff4-8ab7-51303489f11c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:09.006141 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962976d-df37-4ff4-8ab7-51303489f11c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:09.007781 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962976d-df37-4ff4-8ab7-51303489f11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:09.085955 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b962976d-df37-4ff4-8ab7-51303489f11c None None] MFA Rules not processed for user `8121ba4cf2cf44d9a5a55975ea85928c`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:09.087568 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b962976d-df37-4ff4-8ab7-51303489f11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:09.143919 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1424/2843] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:43:08 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:43:09.150627 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d87a8-e5d4-4f56-9590-a84bc267b65b admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:09.182192 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:09.232073 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1420/2844] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 7 13:43:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:43:09.296918 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d87a8-e5d4-4f56-9590-a84bc267b65b admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:09.307783 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8a718-6f38-41e7-8ddd-d3db75ede141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:09.309189 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8a718-6f38-41e7-8ddd-d3db75ede141 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:09.309643 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8a718-6f38-41e7-8ddd-d3db75ede141 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:09.310079 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8a718-6f38-41e7-8ddd-d3db75ede141 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:09.311351 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8a718-6f38-41e7-8ddd-d3db75ede141 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:09.314870 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8a718-6f38-41e7-8ddd-d3db75ede141 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:09.470609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962976d-df37-4ff4-8ab7-51303489f11c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:09.472999 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b962976d-df37-4ff4-8ab7-51303489f11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:09.582345 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:09.592216 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1425/2845] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:43:09 2022] POST /identity/v3/users => generated 292 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:43:09.597248 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8a718-6f38-41e7-8ddd-d3db75ede141 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:09.611320 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8a718-6f38-41e7-8ddd-d3db75ede141 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:09.617097 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522d4bb-1ddd-48ef-b6cf-40c82d312404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:09.617825 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522d4bb-1ddd-48ef-b6cf-40c82d312404 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:09.620620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522d4bb-1ddd-48ef-b6cf-40c82d312404 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:09.620620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522d4bb-1ddd-48ef-b6cf-40c82d312404 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:09.620620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522d4bb-1ddd-48ef-b6cf-40c82d312404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:09.621620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522d4bb-1ddd-48ef-b6cf-40c82d312404 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:09.635661 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1426/2846] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:43:09.641463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962976d-df37-4ff4-8ab7-51303489f11c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:09.642312 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522d4bb-1ddd-48ef-b6cf-40c82d312404 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:09.656857 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522d4bb-1ddd-48ef-b6cf-40c82d312404 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:09.665573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc8600-589b-4fda-82af-5e37319bf79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:09.666169 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc8600-589b-4fda-82af-5e37319bf79a admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:09.666454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc8600-589b-4fda-82af-5e37319bf79a admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:09.666739 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc8600-589b-4fda-82af-5e37319bf79a admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:09.667447 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc8600-589b-4fda-82af-5e37319bf79a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:09.669273 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc8600-589b-4fda-82af-5e37319bf79a admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:09.685094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1427/2847] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:43:09.695275 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc8600-589b-4fda-82af-5e37319bf79a admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:09.701620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc8600-589b-4fda-82af-5e37319bf79a admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:09.707573 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c363c66-fb15-4991-8d68-cdff40f63d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:09.708264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c363c66-fb15-4991-8d68-cdff40f63d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:09.708538 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c363c66-fb15-4991-8d68-cdff40f63d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:09.708995 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c363c66-fb15-4991-8d68-cdff40f63d60 admin admin] PATH_INFO: `/v3/projects/03ab0ddf92ac45188518c000e0ad718b/users/cd225d98dd2b4724b370de27fc595914/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:09.741633 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c363c66-fb15-4991-8d68-cdff40f63d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03ab0ddf92ac45188518c000e0ad718b, user_id=cd225d98dd2b4724b370de27fc595914, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:09.744234 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c363c66-fb15-4991-8d68-cdff40f63d60 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:09.775083 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c1ec6-a338-49c8-a910-a1ed83b812a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cedc7091d2f46b5be68818d26399e9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:09.775731 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c1ec6-a338-49c8-a910-a1ed83b812a4 service swift] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:09.776018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c1ec6-a338-49c8-a910-a1ed83b812a4 service swift] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:09.776293 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c1ec6-a338-49c8-a910-a1ed83b812a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:09.853432 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1428/2848] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:43:09 2022] PUT /identity/v3/projects/03ab0ddf92ac45188518c000e0ad718b/users/cd225d98dd2b4724b370de27fc595914/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:43:09.860716 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da3967-95e1-4a69-acdd-6f7a51d5d886 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:09.861178 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da3967-95e1-4a69-acdd-6f7a51d5d886 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:09.861657 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da3967-95e1-4a69-acdd-6f7a51d5d886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:09.863963 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c1ec6-a338-49c8-a910-a1ed83b812a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:09.867708 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c1ec6-a338-49c8-a910-a1ed83b812a4 service swift] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:09.881793 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1421/2849] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 7 13:43:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 13:43:09.915936 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-42da3967-95e1-4a69-acdd-6f7a51d5d886 None None] MFA Rules not processed for user `cd225d98dd2b4724b370de27fc595914`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:09.915936 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-42da3967-95e1-4a69-acdd-6f7a51d5d886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:10.081254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1429/2850] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:43:10.100618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c1ec6-a338-49c8-a910-a1ed83b812a4 service swift] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:10.103844 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-422c1ec6-a338-49c8-a910-a1ed83b812a4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:10.125576 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:10.257173 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c1ec6-a338-49c8-a910-a1ed83b812a4 service swift] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:10.388582 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af03af8-be67-42f1-b603-20b1128d7bc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:10.389618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af03af8-be67-42f1-b603-20b1128d7bc3 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:10.392080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af03af8-be67-42f1-b603-20b1128d7bc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:10.392080 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af03af8-be67-42f1-b603-20b1128d7bc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:10.407885 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af03af8-be67-42f1-b603-20b1128d7bc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:10.412790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af03af8-be67-42f1-b603-20b1128d7bc3 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:10.431142 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1422/2851] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:43:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:43:15.228651 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da3967-95e1-4a69-acdd-6f7a51d5d886 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:15.374729 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da3967-95e1-4a69-acdd-6f7a51d5d886 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:15.380617 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787d516-3e41-4e1f-91d8-ab1d13af6bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:15.382047 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787d516-3e41-4e1f-91d8-ab1d13af6bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:15.382502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787d516-3e41-4e1f-91d8-ab1d13af6bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:15.383138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787d516-3e41-4e1f-91d8-ab1d13af6bd7 admin admin] PATH_INFO: `/v3/users/cd225d98dd2b4724b370de27fc595914` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:15.385568 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787d516-3e41-4e1f-91d8-ab1d13af6bd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd225d98dd2b4724b370de27fc595914)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:15.386514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787d516-3e41-4e1f-91d8-ab1d13af6bd7 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:15.660493 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:15.663688 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e787d516-3e41-4e1f-91d8-ab1d13af6bd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd225d98dd2b4724b370de27fc595914'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:15.682572 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e787d516-3e41-4e1f-91d8-ab1d13af6bd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd225d98dd2b4724b370de27fc595914'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:15.683562 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e787d516-3e41-4e1f-91d8-ab1d13af6bd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd225d98dd2b4724b370de27fc595914'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:15.704591 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e787d516-3e41-4e1f-91d8-ab1d13af6bd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd225d98dd2b4724b370de27fc595914'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:15.708439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-e787d516-3e41-4e1f-91d8-ab1d13af6bd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd225d98dd2b4724b370de27fc595914'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:15.843702 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1430/2852] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:43:15 2022] DELETE /identity/v3/users/cd225d98dd2b4724b370de27fc595914 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:43:15.853113 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad177f87-e062-428b-85d9-7d4656e830c0 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:15.859808 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad177f87-e062-428b-85d9-7d4656e830c0 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:15.860097 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad177f87-e062-428b-85d9-7d4656e830c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:15.889065 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-ad177f87-e062-428b-85d9-7d4656e830c0 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:15.890594 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ad177f87-e062-428b-85d9-7d4656e830c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:16.007062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1423/2853] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:43:16.019153 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787d516-3e41-4e1f-91d8-ab1d13af6bd7 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:16.022407 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e787d516-3e41-4e1f-91d8-ab1d13af6bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:16.120764 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787d516-3e41-4e1f-91d8-ab1d13af6bd7 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:16.126040 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164c69f-772b-454e-a2c1-9d6539e52737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:16.126812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164c69f-772b-454e-a2c1-9d6539e52737 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:16.127260 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164c69f-772b-454e-a2c1-9d6539e52737 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:16.127638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164c69f-772b-454e-a2c1-9d6539e52737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:16.140540 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164c69f-772b-454e-a2c1-9d6539e52737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:16.143327 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164c69f-772b-454e-a2c1-9d6539e52737 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:16.158165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1431/2854] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:43:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:43:16.746889 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad177f87-e062-428b-85d9-7d4656e830c0 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:16.749656 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ad177f87-e062-428b-85d9-7d4656e830c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:16.766880 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad177f87-e062-428b-85d9-7d4656e830c0 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:16.775126 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5057543-f52d-4eee-83f6-97aa2a994930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:16.775913 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5057543-f52d-4eee-83f6-97aa2a994930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:16.776271 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5057543-f52d-4eee-83f6-97aa2a994930 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:16.776580 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5057543-f52d-4eee-83f6-97aa2a994930 admin admin] PATH_INFO: `/v3/projects/03ab0ddf92ac45188518c000e0ad718b` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:16.779112 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5057543-f52d-4eee-83f6-97aa2a994930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03ab0ddf92ac45188518c000e0ad718b)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:16.781177 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5057543-f52d-4eee-83f6-97aa2a994930 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:16.961409 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-d5057543-f52d-4eee-83f6-97aa2a994930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03ab0ddf92ac45188518c000e0ad718b', 'request_id': 'req-d5057543-f52d-4eee-83f6-97aa2a994930'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:16.968782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-d5057543-f52d-4eee-83f6-97aa2a994930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03ab0ddf92ac45188518c000e0ad718b', 'request_id': 'req-d5057543-f52d-4eee-83f6-97aa2a994930'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:17.627393 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-d5057543-f52d-4eee-83f6-97aa2a994930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03ab0ddf92ac45188518c000e0ad718b', 'request_id': 'req-d5057543-f52d-4eee-83f6-97aa2a994930'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:17.628940 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:17.640429 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-d5057543-f52d-4eee-83f6-97aa2a994930 admin admin] The token cache is being invalidate because project 03ab0ddf92ac45188518c000e0ad718b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:43:17.640723 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-d5057543-f52d-4eee-83f6-97aa2a994930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:17.641522 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-d5057543-f52d-4eee-83f6-97aa2a994930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:17.642954 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1424/2855] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:43:16 2022] DELETE /identity/v3/projects/03ab0ddf92ac45188518c000e0ad718b => generated 0 bytes in 897 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:43:17.649023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6745491-1848-4c22-bdf9-408ae9dbea11 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:17.649595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6745491-1848-4c22-bdf9-408ae9dbea11 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:17.649595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6745491-1848-4c22-bdf9-408ae9dbea11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:17.672660 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c6745491-1848-4c22-bdf9-408ae9dbea11 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:17.674595 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c6745491-1848-4c22-bdf9-408ae9dbea11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:17.745119 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1432/2856] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:43:17.840086 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6d41f-4286-4a42-b17f-401fa9375707 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:17.840488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6d41f-4286-4a42-b17f-401fa9375707 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:17.840790 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6d41f-4286-4a42-b17f-401fa9375707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:17.876918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-5bd6d41f-4286-4a42-b17f-401fa9375707 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:17.879704 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-5bd6d41f-4286-4a42-b17f-401fa9375707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:17.892446 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1425/2857] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:43:17.898855 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6745491-1848-4c22-bdf9-408ae9dbea11 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:17.912394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6745491-1848-4c22-bdf9-408ae9dbea11 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:17.918805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22710e-09f3-42e5-af01-40f93df8a944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:17.919529 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22710e-09f3-42e5-af01-40f93df8a944 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:17.919814 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22710e-09f3-42e5-af01-40f93df8a944 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:17.920193 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22710e-09f3-42e5-af01-40f93df8a944 admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:17.920925 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22710e-09f3-42e5-af01-40f93df8a944 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:43:17.921314 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22710e-09f3-42e5-af01-40f93df8a944 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:17.923266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22710e-09f3-42e5-af01-40f93df8a944 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:17.941381 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1433/2858] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 13:43:17.951797 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6d41f-4286-4a42-b17f-401fa9375707 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:17.963467 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6d41f-4286-4a42-b17f-401fa9375707 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:17.973157 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4424a0-731a-4052-9c89-b09563f2e53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:17.973933 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4424a0-731a-4052-9c89-b09563f2e53c admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:17.974448 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4424a0-731a-4052-9c89-b09563f2e53c admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:17.975132 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4424a0-731a-4052-9c89-b09563f2e53c admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:17.976463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4424a0-731a-4052-9c89-b09563f2e53c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:17.980129 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4424a0-731a-4052-9c89-b09563f2e53c admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:18.137143 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1426/2859] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:43:17 2022] POST /identity/v3/projects => generated 375 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:43:18.144353 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22710e-09f3-42e5-af01-40f93df8a944 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:18.158757 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22710e-09f3-42e5-af01-40f93df8a944 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:18.248187 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:18.263645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee04b3b-0066-4b22-b6f8-058d0152a4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:18.265270 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee04b3b-0066-4b22-b6f8-058d0152a4cc admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:18.265632 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee04b3b-0066-4b22-b6f8-058d0152a4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:18.266067 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee04b3b-0066-4b22-b6f8-058d0152a4cc admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:18.267461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee04b3b-0066-4b22-b6f8-058d0152a4cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:18.270611 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee04b3b-0066-4b22-b6f8-058d0152a4cc admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:18.397798 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1434/2860] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:43:18 2022] POST /identity/v3/users => generated 288 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:43:18.401682 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4424a0-731a-4052-9c89-b09563f2e53c admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:18.420816 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4424a0-731a-4052-9c89-b09563f2e53c admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:18.427332 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26450b5c-7243-4663-873f-35a4690f1cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:18.428734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26450b5c-7243-4663-873f-35a4690f1cae admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:18.429512 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26450b5c-7243-4663-873f-35a4690f1cae admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:18.429915 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26450b5c-7243-4663-873f-35a4690f1cae admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:18.430987 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26450b5c-7243-4663-873f-35a4690f1cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:18.432898 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26450b5c-7243-4663-873f-35a4690f1cae admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:18.450225 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1427/2861] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:43:18.454732 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee04b3b-0066-4b22-b6f8-058d0152a4cc admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:18.470332 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee04b3b-0066-4b22-b6f8-058d0152a4cc admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:18.480233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4012ca-782e-41a5-b523-b554a59beda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:18.480934 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4012ca-782e-41a5-b523-b554a59beda8 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:18.481209 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4012ca-782e-41a5-b523-b554a59beda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:18.481486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4012ca-782e-41a5-b523-b554a59beda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:18.482202 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4012ca-782e-41a5-b523-b554a59beda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:18.483950 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4012ca-782e-41a5-b523-b554a59beda8 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:18.504885 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1435/2862] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:43:18.505942 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26450b5c-7243-4663-873f-35a4690f1cae admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:18.533398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26450b5c-7243-4663-873f-35a4690f1cae admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:18.550154 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4383cd-42a7-4fc7-b2bf-48c0da152fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:18.551394 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4383cd-42a7-4fc7-b2bf-48c0da152fe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:18.551792 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4383cd-42a7-4fc7-b2bf-48c0da152fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:18.552137 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4383cd-42a7-4fc7-b2bf-48c0da152fe6 admin admin] PATH_INFO: `/v3/projects/4e8407f7ed3645e2a15d5f9f08b43e0d/users/9d4eaa0c623240fa8e61d9aad069d7c9/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:18.585219 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4383cd-42a7-4fc7-b2bf-48c0da152fe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8407f7ed3645e2a15d5f9f08b43e0d, user_id=9d4eaa0c623240fa8e61d9aad069d7c9, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:18.590787 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4383cd-42a7-4fc7-b2bf-48c0da152fe6 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:18.717524 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1428/2863] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:43:18 2022] PUT /identity/v3/projects/4e8407f7ed3645e2a15d5f9f08b43e0d/users/9d4eaa0c623240fa8e61d9aad069d7c9/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:43:18.737217 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf08435-20d3-48c0-aa06-ffce9a3f4832 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:18.737944 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf08435-20d3-48c0-aa06-ffce9a3f4832 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:18.737944 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf08435-20d3-48c0-aa06-ffce9a3f4832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:18.792858 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-3bf08435-20d3-48c0-aa06-ffce9a3f4832 None None] MFA Rules not processed for user `9d4eaa0c623240fa8e61d9aad069d7c9`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:18.794281 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-3bf08435-20d3-48c0-aa06-ffce9a3f4832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:18.909145 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1436/2864] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:43:18.921327 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4383cd-42a7-4fc7-b2bf-48c0da152fe6 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:18.925920 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-6a4383cd-42a7-4fc7-b2bf-48c0da152fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:19.046955 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4383cd-42a7-4fc7-b2bf-48c0da152fe6 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:19.052188 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacf838-c938-4cc9-89a6-3d15a97a5b85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:19.053356 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacf838-c938-4cc9-89a6-3d15a97a5b85 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:19.054283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacf838-c938-4cc9-89a6-3d15a97a5b85 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:19.054967 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacf838-c938-4cc9-89a6-3d15a97a5b85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:19.077265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacf838-c938-4cc9-89a6-3d15a97a5b85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:19.087410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacf838-c938-4cc9-89a6-3d15a97a5b85 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:19.107174 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1429/2865] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:43:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:43:20.645521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf08435-20d3-48c0-aa06-ffce9a3f4832 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:20.648853 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-3bf08435-20d3-48c0-aa06-ffce9a3f4832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:20.747416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf08435-20d3-48c0-aa06-ffce9a3f4832 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:20.752805 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eab1a0-d417-4374-ad78-b3eacdf145c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:20.753656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eab1a0-d417-4374-ad78-b3eacdf145c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:20.753932 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eab1a0-d417-4374-ad78-b3eacdf145c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:20.754201 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eab1a0-d417-4374-ad78-b3eacdf145c5 admin admin] PATH_INFO: `/v3/users/2eeb13e580d04add94fbea68be7dcb11` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:20.755599 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eab1a0-d417-4374-ad78-b3eacdf145c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eeb13e580d04add94fbea68be7dcb11)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:20.757970 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eab1a0-d417-4374-ad78-b3eacdf145c5 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:21.425519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-27eab1a0-d417-4374-ad78-b3eacdf145c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eeb13e580d04add94fbea68be7dcb11'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:21.437746 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:21.439738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-27eab1a0-d417-4374-ad78-b3eacdf145c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eeb13e580d04add94fbea68be7dcb11'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:21.440653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-27eab1a0-d417-4374-ad78-b3eacdf145c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eeb13e580d04add94fbea68be7dcb11'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:21.448559 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-27eab1a0-d417-4374-ad78-b3eacdf145c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eeb13e580d04add94fbea68be7dcb11'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:21.449294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-27eab1a0-d417-4374-ad78-b3eacdf145c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eeb13e580d04add94fbea68be7dcb11'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:21.498666 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039bf38-6ccb-45ad-bc53-438fdd7390d7 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:21.499289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039bf38-6ccb-45ad-bc53-438fdd7390d7 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:21.499289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039bf38-6ccb-45ad-bc53-438fdd7390d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:21.527221 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-0039bf38-6ccb-45ad-bc53-438fdd7390d7 None None] MFA Rules not processed for user `9d4eaa0c623240fa8e61d9aad069d7c9`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:21.527221 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0039bf38-6ccb-45ad-bc53-438fdd7390d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:21.534573 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1430/2866] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:43:21.583186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1437/2867] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:43:20 2022] DELETE /identity/v3/users/2eeb13e580d04add94fbea68be7dcb11 => generated 0 bytes in 938 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:43:21.588534 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd770956-db25-4ef3-9756-f1cbb2722b93 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:21.589261 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd770956-db25-4ef3-9756-f1cbb2722b93 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:21.589683 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd770956-db25-4ef3-9756-f1cbb2722b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:21.608005 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eab1a0-d417-4374-ad78-b3eacdf145c5 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:21.612258 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-27eab1a0-d417-4374-ad78-b3eacdf145c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:21.620108 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-dd770956-db25-4ef3-9756-f1cbb2722b93 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:21.622678 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-dd770956-db25-4ef3-9756-f1cbb2722b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:21.732465 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eab1a0-d417-4374-ad78-b3eacdf145c5 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:21.739550 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edbde2-32f8-441b-9ced-736fba0f902f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1a48805d95640489d7ce3333dd687b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:21.740214 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edbde2-32f8-441b-9ced-736fba0f902f service glance] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:21.741043 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edbde2-32f8-441b-9ced-736fba0f902f service glance] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:21.741403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edbde2-32f8-441b-9ced-736fba0f902f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:21.743569 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-65edbde2-32f8-441b-9ced-736fba0f902f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:21.754791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1431/2868] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:43:21.769408 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd770956-db25-4ef3-9756-f1cbb2722b93 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:21.771294 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-dd770956-db25-4ef3-9756-f1cbb2722b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:21.824306 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:21.875459 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd770956-db25-4ef3-9756-f1cbb2722b93 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:21.899165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3f6af-f232-420a-9073-5a4672423b59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:21.899165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3f6af-f232-420a-9073-5a4672423b59 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:21.899165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3f6af-f232-420a-9073-5a4672423b59 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:21.899165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3f6af-f232-420a-9073-5a4672423b59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:21.899586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edbde2-32f8-441b-9ced-736fba0f902f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:21.899586 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edbde2-32f8-441b-9ced-736fba0f902f service glance] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:21.903894 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3f6af-f232-420a-9073-5a4672423b59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:21.905681 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3f6af-f232-420a-9073-5a4672423b59 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:21.927647 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1438/2869] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:43:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:43:21.941878 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1432/2870] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:43:22.494705 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edbde2-32f8-441b-9ced-736fba0f902f service glance] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:22.495947 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-65edbde2-32f8-441b-9ced-736fba0f902f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:22.514838 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edbde2-32f8-441b-9ced-736fba0f902f service glance] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:22.519970 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f920337-1e74-4cfc-a3f1-a121f6877adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:22.520808 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f920337-1e74-4cfc-a3f1-a121f6877adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:22.521191 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f920337-1e74-4cfc-a3f1-a121f6877adb admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:22.521537 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f920337-1e74-4cfc-a3f1-a121f6877adb admin admin] PATH_INFO: `/v3/projects/f5bb7647c9954ad6a54763758dfb117f` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:22.523841 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f920337-1e74-4cfc-a3f1-a121f6877adb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5bb7647c9954ad6a54763758dfb117f)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:22.525781 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f920337-1e74-4cfc-a3f1-a121f6877adb admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:22.660821 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3f6af-f232-420a-9073-5a4672423b59 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:22.663344 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-2ff3f6af-f232-420a-9073-5a4672423b59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:22.722502 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9f920337-1e74-4cfc-a3f1-a121f6877adb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5bb7647c9954ad6a54763758dfb117f', 'request_id': 'req-9f920337-1e74-4cfc-a3f1-a121f6877adb'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:22.730812 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9f920337-1e74-4cfc-a3f1-a121f6877adb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5bb7647c9954ad6a54763758dfb117f', 'request_id': 'req-9f920337-1e74-4cfc-a3f1-a121f6877adb'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:22.746164 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3f6af-f232-420a-9073-5a4672423b59 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:22.810396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9f920337-1e74-4cfc-a3f1-a121f6877adb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5bb7647c9954ad6a54763758dfb117f', 'request_id': 'req-9f920337-1e74-4cfc-a3f1-a121f6877adb'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:22.820331 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4454b3af-9509-4c39-93eb-4cbb330f2728 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bee90bf951f04c2e8e19ceeacb8cfcb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:22.820959 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454b3af-9509-4c39-93eb-4cbb330f2728 service placement] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:22.821791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454b3af-9509-4c39-93eb-4cbb330f2728 service placement] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:22.821791 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454b3af-9509-4c39-93eb-4cbb330f2728 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:22.823334 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-4454b3af-9509-4c39-93eb-4cbb330f2728 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:22.835660 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9f920337-1e74-4cfc-a3f1-a121f6877adb admin admin] The token cache is being invalidate because project f5bb7647c9954ad6a54763758dfb117f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:43:22.843754 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9f920337-1e74-4cfc-a3f1-a121f6877adb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:22.843921 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4454b3af-9509-4c39-93eb-4cbb330f2728 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:22.844018 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-9f920337-1e74-4cfc-a3f1-a121f6877adb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:22.844120 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4454b3af-9509-4c39-93eb-4cbb330f2728 service placement] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:22.844221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1439/2871] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:43:22 2022] DELETE /identity/v3/projects/f5bb7647c9954ad6a54763758dfb117f => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:43:22.844221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f920337-1e74-4cfc-a3f1-a121f6877adb admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:22.844221 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-9f920337-1e74-4cfc-a3f1-a121f6877adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:22.849229 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:22.861619 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1433/2872] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:43:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:43:22.934651 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f920337-1e74-4cfc-a3f1-a121f6877adb admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:22.939148 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b699cc-d374-4e53-b95e-9b177a6a5039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:22.940267 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b699cc-d374-4e53-b95e-9b177a6a5039 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:22.940745 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b699cc-d374-4e53-b95e-9b177a6a5039 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:22.941210 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b699cc-d374-4e53-b95e-9b177a6a5039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:22.944460 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-65b699cc-d374-4e53-b95e-9b177a6a5039 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:22.999154 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927e801-1e03-4d8e-a5ed-27b3520ec976 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:22.999469 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927e801-1e03-4d8e-a5ed-27b3520ec976 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:22.999763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927e801-1e03-4d8e-a5ed-27b3520ec976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:23.026854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-b927e801-1e03-4d8e-a5ed-27b3520ec976 None None] MFA Rules not processed for user `8895cee0f4444866bd4c3908eb248543`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:23.028583 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b927e801-1e03-4d8e-a5ed-27b3520ec976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:23.053977 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b699cc-d374-4e53-b95e-9b177a6a5039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:23.053977 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b699cc-d374-4e53-b95e-9b177a6a5039 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:23.080253 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1440/2873] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:43:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:43:23.110290 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1434/2874] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:43:28.018048 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b699cc-d374-4e53-b95e-9b177a6a5039 service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:28.027648 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b699cc-d374-4e53-b95e-9b177a6a5039 service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:28.038172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2720430-0e08-49e7-82c0-bbf92d30d12d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:28.038172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2720430-0e08-49e7-82c0-bbf92d30d12d service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:28.038172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2720430-0e08-49e7-82c0-bbf92d30d12d service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:28.038172 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2720430-0e08-49e7-82c0-bbf92d30d12d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:28.039131 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f2720430-0e08-49e7-82c0-bbf92d30d12d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:28.046551 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2720430-0e08-49e7-82c0-bbf92d30d12d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:28.048045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2720430-0e08-49e7-82c0-bbf92d30d12d service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:28.058412 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1441/2875] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:43:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:43:28.496583 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec6843-1458-4d3c-b592-c7466b2d7f8c None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:28.497107 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec6843-1458-4d3c-b592-c7466b2d7f8c None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:28.497240 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec6843-1458-4d3c-b592-c7466b2d7f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:28.505634 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=100 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:28.520240 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-66ec6843-1458-4d3c-b592-c7466b2d7f8c None None] MFA Rules not processed for user `8121ba4cf2cf44d9a5a55975ea85928c`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:28.521919 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-66ec6843-1458-4d3c-b592-c7466b2d7f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:28.591680 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1435/2876] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 7 13:43:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:43:28.604713 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2720430-0e08-49e7-82c0-bbf92d30d12d service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:28.606165 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f2720430-0e08-49e7-82c0-bbf92d30d12d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:28.689045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2720430-0e08-49e7-82c0-bbf92d30d12d service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:28.694298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d8a17-5efb-4945-b270-1cb234ff6022 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cedc7091d2f46b5be68818d26399e9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:28.695480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d8a17-5efb-4945-b270-1cb234ff6022 service swift] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:28.695480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d8a17-5efb-4945-b270-1cb234ff6022 service swift] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:28.695480 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d8a17-5efb-4945-b270-1cb234ff6022 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:28.706380 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d8a17-5efb-4945-b270-1cb234ff6022 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:28.708362 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d8a17-5efb-4945-b270-1cb234ff6022 service swift] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:28.720959 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1442/2877] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 7 13:43:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 13:43:33.699184 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec6843-1458-4d3c-b592-c7466b2d7f8c None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:33.700931 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-66ec6843-1458-4d3c-b592-c7466b2d7f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:33.801886 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec6843-1458-4d3c-b592-c7466b2d7f8c None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:33.807895 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa850f-fffd-4910-b590-801e51aef5d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc2fa0fa46b842ceb1cac437166b8aea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:33.808779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa850f-fffd-4910-b590-801e51aef5d4 service cinder] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:33.809156 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa850f-fffd-4910-b590-801e51aef5d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:33.809468 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa850f-fffd-4910-b590-801e51aef5d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:33.811845 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-2baa850f-fffd-4910-b590-801e51aef5d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:33.987258 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa850f-fffd-4910-b590-801e51aef5d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:33.999287 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa850f-fffd-4910-b590-801e51aef5d4 service cinder] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:34.023657 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1436/2878] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:43:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:43:38.716867 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=90 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:44.016248 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=72 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:43:44.329518 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b54c3f-486d-4c55-8e8a-aee1d7650e93 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:44.330266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b54c3f-486d-4c55-8e8a-aee1d7650e93 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:44.330266 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b54c3f-486d-4c55-8e8a-aee1d7650e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:44.350575 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-f0b54c3f-486d-4c55-8e8a-aee1d7650e93 None None] MFA Rules not processed for user `9dae215cd92f46d18a3c1b7389c3af4b`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:44.351925 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f0b54c3f-486d-4c55-8e8a-aee1d7650e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:44.359311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1443/2879] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:43:49.888371 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa850f-fffd-4910-b590-801e51aef5d4 service cinder] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:49.892409 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-2baa850f-fffd-4910-b590-801e51aef5d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:49.932597 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa850f-fffd-4910-b590-801e51aef5d4 service cinder] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:49.942813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c7899-f654-4116-b8f9-804dd78d065f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:49.943899 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c7899-f654-4116-b8f9-804dd78d065f service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:49.944334 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c7899-f654-4116-b8f9-804dd78d065f service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:49.944726 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c7899-f654-4116-b8f9-804dd78d065f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:49.948038 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ba2c7899-f654-4116-b8f9-804dd78d065f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:49.968056 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c7899-f654-4116-b8f9-804dd78d065f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:49.972260 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c7899-f654-4116-b8f9-804dd78d065f service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:49.989601 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1437/2880] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:43:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:43:52.504571 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a9c24-4e5d-473a-9f73-759d0fecd006 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:52.505340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a9c24-4e5d-473a-9f73-759d0fecd006 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:52.505340 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a9c24-4e5d-473a-9f73-759d0fecd006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:52.532082 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-ac2a9c24-4e5d-473a-9f73-759d0fecd006 None None] MFA Rules not processed for user `8121ba4cf2cf44d9a5a55975ea85928c`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:52.533887 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-ac2a9c24-4e5d-473a-9f73-759d0fecd006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:52.543257 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1444/2881] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 7 13:43:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:43:52.564874 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c7899-f654-4116-b8f9-804dd78d065f service nova] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:52.579995 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c7899-f654-4116-b8f9-804dd78d065f service nova] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:52.585618 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de4a89-9d47-4847-9589-a9ca1b6963eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cedc7091d2f46b5be68818d26399e9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:52.586453 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de4a89-9d47-4847-9589-a9ca1b6963eb service swift] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:52.586730 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de4a89-9d47-4847-9589-a9ca1b6963eb service swift] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:52.587057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de4a89-9d47-4847-9589-a9ca1b6963eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:52.597062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de4a89-9d47-4847-9589-a9ca1b6963eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:52.599410 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de4a89-9d47-4847-9589-a9ca1b6963eb service swift] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:52.610710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1438/2882] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 7 13:43:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:43:54.160753 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6e834-e522-43ad-b69e-d412da33b590 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:54.162244 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6e834-e522-43ad-b69e-d412da33b590 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:54.162773 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6e834-e522-43ad-b69e-d412da33b590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:54.189910 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-39e6e834-e522-43ad-b69e-d412da33b590 None None] MFA Rules not processed for user `8121ba4cf2cf44d9a5a55975ea85928c`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:54.191997 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-39e6e834-e522-43ad-b69e-d412da33b590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:54.206791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1445/2883] 10.222.0.62 () {58 vars in 967 bytes} [Wed Sep 7 13:43:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:43:54.229441 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de4a89-9d47-4847-9589-a9ca1b6963eb service swift] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:54.249399 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de4a89-9d47-4847-9589-a9ca1b6963eb service swift] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:54.256985 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9112c9e-b808-4f7b-a441-ee52efa357d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cedc7091d2f46b5be68818d26399e9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:54.258484 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9112c9e-b808-4f7b-a441-ee52efa357d0 service swift] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:54.259669 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9112c9e-b808-4f7b-a441-ee52efa357d0 service swift] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:54.260075 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9112c9e-b808-4f7b-a441-ee52efa357d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:54.272939 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9112c9e-b808-4f7b-a441-ee52efa357d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:54.276031 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9112c9e-b808-4f7b-a441-ee52efa357d0 service swift] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:54.294234 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1439/2884] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 7 13:43:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:43:55.227964 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aa1dc-665b-42ec-a54f-2d0bdc852971 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:55.228470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aa1dc-665b-42ec-a54f-2d0bdc852971 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:55.229025 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aa1dc-665b-42ec-a54f-2d0bdc852971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:55.258484 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-892aa1dc-665b-42ec-a54f-2d0bdc852971 None None] MFA Rules not processed for user `8121ba4cf2cf44d9a5a55975ea85928c`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:55.259725 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-892aa1dc-665b-42ec-a54f-2d0bdc852971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:55.267337 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1446/2885] 10.222.0.62 () {58 vars in 1009 bytes} [Wed Sep 7 13:43:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:43:55.281232 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9112c9e-b808-4f7b-a441-ee52efa357d0 service swift] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:55.294813 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9112c9e-b808-4f7b-a441-ee52efa357d0 service swift] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:55.300714 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130fb611-0e3d-43ca-8963-7a900d6631dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cedc7091d2f46b5be68818d26399e9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:55.301389 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fb611-0e3d-43ca-8963-7a900d6631dc service swift] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:55.301705 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fb611-0e3d-43ca-8963-7a900d6631dc service swift] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:55.302092 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fb611-0e3d-43ca-8963-7a900d6631dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:55.314165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130fb611-0e3d-43ca-8963-7a900d6631dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:55.316709 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130fb611-0e3d-43ca-8963-7a900d6631dc service swift] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:55.327342 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1440/2886] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 7 13:43:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:43:56.196541 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590b4be-58c9-4623-a52b-ec5f9271768e None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:56.197411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590b4be-58c9-4623-a52b-ec5f9271768e None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:56.197966 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590b4be-58c9-4623-a52b-ec5f9271768e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:56.225272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-c590b4be-58c9-4623-a52b-ec5f9271768e None None] MFA Rules not processed for user `8121ba4cf2cf44d9a5a55975ea85928c`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:56.226703 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c590b4be-58c9-4623-a52b-ec5f9271768e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:56.236379 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1447/2887] 10.222.0.62 () {58 vars in 967 bytes} [Wed Sep 7 13:43:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:43:56.250370 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130fb611-0e3d-43ca-8963-7a900d6631dc service swift] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:56.263698 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130fb611-0e3d-43ca-8963-7a900d6631dc service swift] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:56.269370 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28def447-9374-40c3-af6b-96998439f637 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cedc7091d2f46b5be68818d26399e9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:56.269997 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28def447-9374-40c3-af6b-96998439f637 service swift] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:56.270289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28def447-9374-40c3-af6b-96998439f637 service swift] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:56.270549 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28def447-9374-40c3-af6b-96998439f637 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:56.285746 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28def447-9374-40c3-af6b-96998439f637 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:56.288935 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28def447-9374-40c3-af6b-96998439f637 service swift] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:56.307289 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1441/2888] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 7 13:43:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:43:59.526893 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590b4be-58c9-4623-a52b-ec5f9271768e None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:43:59.528886 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c590b4be-58c9-4623-a52b-ec5f9271768e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:43:59.615176 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590b4be-58c9-4623-a52b-ec5f9271768e None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:43:59.620615 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c19a46-3e31-4f26-bd83-afec58d7b700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:43:59.621201 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c19a46-3e31-4f26-bd83-afec58d7b700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:59.621486 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c19a46-3e31-4f26-bd83-afec58d7b700 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:59.621772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c19a46-3e31-4f26-bd83-afec58d7b700 admin admin] PATH_INFO: `/v3/users/9d4eaa0c623240fa8e61d9aad069d7c9` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:59.624183 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c19a46-3e31-4f26-bd83-afec58d7b700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d4eaa0c623240fa8e61d9aad069d7c9)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:43:59.626493 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c19a46-3e31-4f26-bd83-afec58d7b700 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:43:59.857166 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-29c19a46-3e31-4f26-bd83-afec58d7b700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d4eaa0c623240fa8e61d9aad069d7c9'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:59.872289 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-29c19a46-3e31-4f26-bd83-afec58d7b700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d4eaa0c623240fa8e61d9aad069d7c9'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:59.873134 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-29c19a46-3e31-4f26-bd83-afec58d7b700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d4eaa0c623240fa8e61d9aad069d7c9'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:59.886346 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-29c19a46-3e31-4f26-bd83-afec58d7b700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d4eaa0c623240fa8e61d9aad069d7c9'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:59.887772 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-29c19a46-3e31-4f26-bd83-afec58d7b700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d4eaa0c623240fa8e61d9aad069d7c9'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:43:59.964988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1448/2889] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:43:59 2022] DELETE /identity/v3/users/9d4eaa0c623240fa8e61d9aad069d7c9 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:43:59.969683 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff459e67-f38e-4728-b625-f4643970d18f None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:43:59.970365 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff459e67-f38e-4728-b625-f4643970d18f None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:43:59.970365 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff459e67-f38e-4728-b625-f4643970d18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:43:59.991832 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-ff459e67-f38e-4728-b625-f4643970d18f None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:43:59.993212 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ff459e67-f38e-4728-b625-f4643970d18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:44:00.062016 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1442/2890] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:44:00.074551 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c19a46-3e31-4f26-bd83-afec58d7b700 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:00.076319 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-29c19a46-3e31-4f26-bd83-afec58d7b700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:44:00.149672 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:44:00.156078 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c19a46-3e31-4f26-bd83-afec58d7b700 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:00.159816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7daeef-664f-409d-a62d-05eaa177eba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:00.163138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7daeef-664f-409d-a62d-05eaa177eba7 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:00.163138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7daeef-664f-409d-a62d-05eaa177eba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:00.163138 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7daeef-664f-409d-a62d-05eaa177eba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:00.175141 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7daeef-664f-409d-a62d-05eaa177eba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:00.177683 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7daeef-664f-409d-a62d-05eaa177eba7 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:00.194027 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1449/2891] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:44:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:44:00.552295 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff459e67-f38e-4728-b625-f4643970d18f None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:00.553736 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-ff459e67-f38e-4728-b625-f4643970d18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:44:00.569313 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff459e67-f38e-4728-b625-f4643970d18f None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:00.576116 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e93024-48cb-4049-9bd4-21de2a3f366d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:00.576847 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e93024-48cb-4049-9bd4-21de2a3f366d admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:00.577197 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e93024-48cb-4049-9bd4-21de2a3f366d admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:00.577511 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e93024-48cb-4049-9bd4-21de2a3f366d admin admin] PATH_INFO: `/v3/projects/4e8407f7ed3645e2a15d5f9f08b43e0d` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:00.579919 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e93024-48cb-4049-9bd4-21de2a3f366d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e8407f7ed3645e2a15d5f9f08b43e0d)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:00.582263 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e93024-48cb-4049-9bd4-21de2a3f366d admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:00.712146 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-f8e93024-48cb-4049-9bd4-21de2a3f366d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e8407f7ed3645e2a15d5f9f08b43e0d', 'request_id': 'req-f8e93024-48cb-4049-9bd4-21de2a3f366d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:44:00.718455 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-f8e93024-48cb-4049-9bd4-21de2a3f366d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e8407f7ed3645e2a15d5f9f08b43e0d', 'request_id': 'req-f8e93024-48cb-4049-9bd4-21de2a3f366d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:44:00.843798 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-f8e93024-48cb-4049-9bd4-21de2a3f366d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e8407f7ed3645e2a15d5f9f08b43e0d', 'request_id': 'req-f8e93024-48cb-4049-9bd4-21de2a3f366d'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:44:00.862122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-f8e93024-48cb-4049-9bd4-21de2a3f366d admin admin] The token cache is being invalidate because project 4e8407f7ed3645e2a15d5f9f08b43e0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:44:00.862122 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-f8e93024-48cb-4049-9bd4-21de2a3f366d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:44:00.863777 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-f8e93024-48cb-4049-9bd4-21de2a3f366d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:44:00.867211 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1443/2892] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:44:00 2022] DELETE /identity/v3/projects/4e8407f7ed3645e2a15d5f9f08b43e0d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:44:00.874634 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ced32-6e9a-46e6-b157-3a09982ae478 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:00.875264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ced32-6e9a-46e6-b157-3a09982ae478 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:00.875264 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ced32-6e9a-46e6-b157-3a09982ae478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:00.911083 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-e12ced32-6e9a-46e6-b157-3a09982ae478 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:44:00.913228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-e12ced32-6e9a-46e6-b157-3a09982ae478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:44:01.000346 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1450/2893] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:44:01.008685 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc769de-dd84-495d-88a2-87c20058bb56 None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:01.009319 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc769de-dd84-495d-88a2-87c20058bb56 None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:01.009319 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc769de-dd84-495d-88a2-87c20058bb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:01.035329 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-8bc769de-dd84-495d-88a2-87c20058bb56 None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:44:01.040918 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8bc769de-dd84-495d-88a2-87c20058bb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:44:01.048303 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1444/2894] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 13:44:01.053517 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ced32-6e9a-46e6-b157-3a09982ae478 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:01.067038 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ced32-6e9a-46e6-b157-3a09982ae478 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:01.073542 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a761f3c-a647-487e-8f96-c351c8e360ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:01.074401 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a761f3c-a647-487e-8f96-c351c8e360ef admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:01.074827 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a761f3c-a647-487e-8f96-c351c8e360ef admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:01.075395 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a761f3c-a647-487e-8f96-c351c8e360ef admin admin] PATH_INFO: `/v3/domains` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:01.076425 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a761f3c-a647-487e-8f96-c351c8e360ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=105316) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 13:44:01.076776 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a761f3c-a647-487e-8f96-c351c8e360ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:01.078781 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a761f3c-a647-487e-8f96-c351c8e360ef admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:01.093411 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1451/2895] 10.222.0.62 () {58 vars in 1153 bytes} [Wed Sep 7 13:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 13:44:01.100417 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc769de-dd84-495d-88a2-87c20058bb56 None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:01.111195 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc769de-dd84-495d-88a2-87c20058bb56 None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:01.116150 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840aae9-1c5d-4dc1-99b2-8ec7edf99b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:01.117283 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840aae9-1c5d-4dc1-99b2-8ec7edf99b97 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:01.117560 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840aae9-1c5d-4dc1-99b2-8ec7edf99b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:01.117839 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840aae9-1c5d-4dc1-99b2-8ec7edf99b97 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:01.118607 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840aae9-1c5d-4dc1-99b2-8ec7edf99b97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:01.120564 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840aae9-1c5d-4dc1-99b2-8ec7edf99b97 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:01.131217 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:44:01.284687 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1445/2896] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:44:01 2022] POST /identity/v3/projects => generated 371 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:44:01.290442 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a761f3c-a647-487e-8f96-c351c8e360ef admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:01.301271 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a761f3c-a647-487e-8f96-c351c8e360ef admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:01.366693 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac6530-1f87-4b66-bc69-ee11bc893f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:01.367809 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac6530-1f87-4b66-bc69-ee11bc893f47 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:01.368091 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac6530-1f87-4b66-bc69-ee11bc893f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:01.368470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac6530-1f87-4b66-bc69-ee11bc893f47 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:01.369150 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac6530-1f87-4b66-bc69-ee11bc893f47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:01.371225 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac6530-1f87-4b66-bc69-ee11bc893f47 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:01.472051 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1452/2897] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:44:01 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 13:44:01.477600 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840aae9-1c5d-4dc1-99b2-8ec7edf99b97 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:01.490754 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840aae9-1c5d-4dc1-99b2-8ec7edf99b97 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:01.498780 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217d19b-d6c6-4dde-819a-0ad68bbfb218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:01.499938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217d19b-d6c6-4dde-819a-0ad68bbfb218 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:01.500875 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217d19b-d6c6-4dde-819a-0ad68bbfb218 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:01.502045 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217d19b-d6c6-4dde-819a-0ad68bbfb218 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:01.503763 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217d19b-d6c6-4dde-819a-0ad68bbfb218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:01.506905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217d19b-d6c6-4dde-819a-0ad68bbfb218 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:01.526692 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1446/2898] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:44:01.531777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac6530-1f87-4b66-bc69-ee11bc893f47 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:01.545749 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac6530-1f87-4b66-bc69-ee11bc893f47 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:01.560308 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a38a2-ddc3-4d6c-af8d-5f21070e3771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:01.561466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a38a2-ddc3-4d6c-af8d-5f21070e3771 admin admin] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:01.561789 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a38a2-ddc3-4d6c-af8d-5f21070e3771 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:01.562194 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a38a2-ddc3-4d6c-af8d-5f21070e3771 admin admin] PATH_INFO: `/v3/roles` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:01.563292 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a38a2-ddc3-4d6c-af8d-5f21070e3771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:01.566052 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a38a2-ddc3-4d6c-af8d-5f21070e3771 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:01.582552 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1453/2899] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:44:01.588938 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217d19b-d6c6-4dde-819a-0ad68bbfb218 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:01.606835 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217d19b-d6c6-4dde-819a-0ad68bbfb218 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:01.615680 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4673ad-f9d7-417e-9d5e-d2c49916c061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:01.616377 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4673ad-f9d7-417e-9d5e-d2c49916c061 admin admin] REQUEST_METHOD: `PUT` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:01.617423 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4673ad-f9d7-417e-9d5e-d2c49916c061 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:01.618293 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4673ad-f9d7-417e-9d5e-d2c49916c061 admin admin] PATH_INFO: `/v3/projects/bb7b613bf3c94b4aa49f450be9971774/users/138facb7768545999dd43217600771a3/roles/bcc52f73cf03442e922a4757908334de` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:01.638932 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4673ad-f9d7-417e-9d5e-d2c49916c061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb7b613bf3c94b4aa49f450be9971774, user_id=138facb7768545999dd43217600771a3, role_id=bcc52f73cf03442e922a4757908334de)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:01.641934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4673ad-f9d7-417e-9d5e-d2c49916c061 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:01.784296 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1447/2900] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:44:01 2022] PUT /identity/v3/projects/bb7b613bf3c94b4aa49f450be9971774/users/138facb7768545999dd43217600771a3/roles/bcc52f73cf03442e922a4757908334de => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:44:01.789810 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3f01e-3070-4c1e-80af-f5a699708abc None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:01.790094 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3f01e-3070-4c1e-80af-f5a699708abc None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:01.790449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3f01e-3070-4c1e-80af-f5a699708abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:01.823767 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-42e3f01e-3070-4c1e-80af-f5a699708abc None None] MFA Rules not processed for user `7a8e484a77ce40f880d21a33a930f0be`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:44:01.825096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-42e3f01e-3070-4c1e-80af-f5a699708abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:44:01.898638 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:44:01.920254 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1454/2901] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:44:01.933050 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4673ad-f9d7-417e-9d5e-d2c49916c061 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:01.934848 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-8f4673ad-f9d7-417e-9d5e-d2c49916c061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:44:02.008912 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4673ad-f9d7-417e-9d5e-d2c49916c061 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:02.012782 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d7203-3990-4d0a-be6b-9aed98671ac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:02.013390 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d7203-3990-4d0a-be6b-9aed98671ac1 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:02.013673 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d7203-3990-4d0a-be6b-9aed98671ac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:02.013945 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d7203-3990-4d0a-be6b-9aed98671ac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:02.024172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d7203-3990-4d0a-be6b-9aed98671ac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:02.025690 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d7203-3990-4d0a-be6b-9aed98671ac1 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:02.038958 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1448/2902] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:44:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:44:03.054792 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3f01e-3070-4c1e-80af-f5a699708abc None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:03.056449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-42e3f01e-3070-4c1e-80af-f5a699708abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:44:03.070591 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3f01e-3070-4c1e-80af-f5a699708abc None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:03.074916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6e2c2-db6b-441f-9ab8-f14f19d96b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:03.075577 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6e2c2-db6b-441f-9ab8-f14f19d96b36 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:03.076042 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6e2c2-db6b-441f-9ab8-f14f19d96b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:03.076561 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6e2c2-db6b-441f-9ab8-f14f19d96b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:03.078597 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-a0c6e2c2-db6b-441f-9ab8-f14f19d96b36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:44:03.171644 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6e2c2-db6b-441f-9ab8-f14f19d96b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:03.173183 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6e2c2-db6b-441f-9ab8-f14f19d96b36 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:03.185930 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1455/2903] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:44:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:44:05.337241 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d7203-3990-4d0a-be6b-9aed98671ac1 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:05.338481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e04d7203-3990-4d0a-be6b-9aed98671ac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:44:05.610936 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d7203-3990-4d0a-be6b-9aed98671ac1 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:05.629866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b0aad-7c2e-4141-a68b-0c625a26067f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:05.629866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b0aad-7c2e-4141-a68b-0c625a26067f service nova] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:05.629866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b0aad-7c2e-4141-a68b-0c625a26067f service nova] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:05.629866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b0aad-7c2e-4141-a68b-0c625a26067f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:05.630358 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-739b0aad-7c2e-4141-a68b-0c625a26067f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:44:05.645982 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b0aad-7c2e-4141-a68b-0c625a26067f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:05.648592 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b0aad-7c2e-4141-a68b-0c625a26067f service nova] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:05.662897 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1449/2904] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:44:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:44:06.283293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6e2c2-db6b-441f-9ab8-f14f19d96b36 service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:06.301501 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6e2c2-db6b-441f-9ab8-f14f19d96b36 service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:06.313023 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cec2fb-51ef-4462-8b12-0cfa9ce61891 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:06.313965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cec2fb-51ef-4462-8b12-0cfa9ce61891 service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:06.314419 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cec2fb-51ef-4462-8b12-0cfa9ce61891 service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:06.314697 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cec2fb-51ef-4462-8b12-0cfa9ce61891 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:06.321095 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-19cec2fb-51ef-4462-8b12-0cfa9ce61891 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:44:06.346033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cec2fb-51ef-4462-8b12-0cfa9ce61891 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:06.351806 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cec2fb-51ef-4462-8b12-0cfa9ce61891 service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:06.371384 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1456/2905] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:44:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:44:09.760799 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b0aad-7c2e-4141-a68b-0c625a26067f service nova] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:09.766630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b0aad-7c2e-4141-a68b-0c625a26067f service nova] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:09.772705 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059ec8b-6224-46de-ba12-5ccb9b18d2f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:09.774093 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059ec8b-6224-46de-ba12-5ccb9b18d2f0 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:09.774602 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059ec8b-6224-46de-ba12-5ccb9b18d2f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:09.775102 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059ec8b-6224-46de-ba12-5ccb9b18d2f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:09.777863 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c059ec8b-6224-46de-ba12-5ccb9b18d2f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:44:09.848825 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:44:09.886213 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c059ec8b-6224-46de-ba12-5ccb9b18d2f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:09.888020 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c059ec8b-6224-46de-ba12-5ccb9b18d2f0 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:09.902649 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1450/2906] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:44:09 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:44:11.893620 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4047d-fdc1-4e28-bed3-f010e8cdb2aa None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:11.894393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4047d-fdc1-4e28-bed3-f010e8cdb2aa None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:11.894393 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4047d-fdc1-4e28-bed3-f010e8cdb2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:11.958683 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-aec4047d-fdc1-4e28-bed3-f010e8cdb2aa None None] MFA Rules not processed for user `138facb7768545999dd43217600771a3`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:44:11.960044 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-aec4047d-fdc1-4e28-bed3-f010e8cdb2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:44:12.170710 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1457/2907] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:44:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:44:12.175542 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059ec8b-6224-46de-ba12-5ccb9b18d2f0 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:12.190498 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059ec8b-6224-46de-ba12-5ccb9b18d2f0 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:12.206172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48a5a7-b577-4f62-83f2-76700ccb5863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:12.207458 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48a5a7-b577-4f62-83f2-76700ccb5863 admin admin] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:12.208438 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48a5a7-b577-4f62-83f2-76700ccb5863 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:12.209278 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48a5a7-b577-4f62-83f2-76700ccb5863 admin admin] PATH_INFO: `/v3/projects` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:12.211590 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48a5a7-b577-4f62-83f2-76700ccb5863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:12.214286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48a5a7-b577-4f62-83f2-76700ccb5863 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:12.346488 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1451/2908] 10.222.0.62 () {60 vars in 1153 bytes} [Wed Sep 7 13:44:12 2022] POST /identity/v3/projects => generated 371 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:44:12.354777 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4047d-fdc1-4e28-bed3-f010e8cdb2aa None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:12.381152 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4047d-fdc1-4e28-bed3-f010e8cdb2aa None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:12.537285 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e4608-6603-4ccf-a301-1cb7c0a08285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:12.539290 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e4608-6603-4ccf-a301-1cb7c0a08285 admin admin] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:12.539782 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e4608-6603-4ccf-a301-1cb7c0a08285 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:12.540519 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e4608-6603-4ccf-a301-1cb7c0a08285 admin admin] PATH_INFO: `/v3/users` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:12.542008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e4608-6603-4ccf-a301-1cb7c0a08285 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:12.546035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e4608-6603-4ccf-a301-1cb7c0a08285 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:12.582782 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:44:12.711776 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48a5a7-b577-4f62-83f2-76700ccb5863 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:12.712207 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1458/2909] 10.222.0.62 () {60 vars in 1144 bytes} [Wed Sep 7 13:44:12 2022] POST /identity/v3/users => generated 292 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 13:44:12.730133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48a5a7-b577-4f62-83f2-76700ccb5863 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:12.739226 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0677e4-98cd-468e-b78d-eef253ceb6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:12.739226 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0677e4-98cd-468e-b78d-eef253ceb6c5 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:12.739226 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0677e4-98cd-468e-b78d-eef253ceb6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:12.739226 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0677e4-98cd-468e-b78d-eef253ceb6c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:12.741686 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0677e4-98cd-468e-b78d-eef253ceb6c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:12.743824 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0677e4-98cd-468e-b78d-eef253ceb6c5 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:12.763113 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1452/2910] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 13:44:12.768794 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e4608-6603-4ccf-a301-1cb7c0a08285 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:12.787151 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e4608-6603-4ccf-a301-1cb7c0a08285 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:12.793711 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b891c3-deb6-484a-bbd2-1a7b5a7cdd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:12.794412 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b891c3-deb6-484a-bbd2-1a7b5a7cdd14 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:12.794692 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b891c3-deb6-484a-bbd2-1a7b5a7cdd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:12.795233 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b891c3-deb6-484a-bbd2-1a7b5a7cdd14 admin admin] PATH_INFO: `/v3/projects/66d57cf1e2814541bd4a9f56e86a42c9/users/0bb14422779947f6a6efb0cd19bd1d3f/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:12.840816 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b891c3-deb6-484a-bbd2-1a7b5a7cdd14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66d57cf1e2814541bd4a9f56e86a42c9, user_id=0bb14422779947f6a6efb0cd19bd1d3f, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:12.846166 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b891c3-deb6-484a-bbd2-1a7b5a7cdd14 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:12.986045 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1459/2911] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:44:12 2022] PUT /identity/v3/projects/66d57cf1e2814541bd4a9f56e86a42c9/users/0bb14422779947f6a6efb0cd19bd1d3f/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:44:12.992902 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0677e4-98cd-468e-b78d-eef253ceb6c5 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:13.010630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0677e4-98cd-468e-b78d-eef253ceb6c5 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:13.176929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192c83c-75ba-4641-9e2f-11d3da647308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:13.178020 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192c83c-75ba-4641-9e2f-11d3da647308 admin admin] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:13.178456 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192c83c-75ba-4641-9e2f-11d3da647308 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:13.184186 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192c83c-75ba-4641-9e2f-11d3da647308 admin admin] PATH_INFO: `/v3/roles` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:13.194099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192c83c-75ba-4641-9e2f-11d3da647308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:13.194099 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192c83c-75ba-4641-9e2f-11d3da647308 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:13.234581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1453/2912] 10.222.0.62 () {58 vars in 1122 bytes} [Wed Sep 7 13:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 13:44:13.236167 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b891c3-deb6-484a-bbd2-1a7b5a7cdd14 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:13.254883 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b891c3-deb6-484a-bbd2-1a7b5a7cdd14 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:13.264037 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1baca4-c6fc-41ff-865b-5f9e1b77c6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:13.268294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1baca4-c6fc-41ff-865b-5f9e1b77c6d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:13.268294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1baca4-c6fc-41ff-865b-5f9e1b77c6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:13.268294 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1baca4-c6fc-41ff-865b-5f9e1b77c6d5 admin admin] PATH_INFO: `/v3/projects/66d57cf1e2814541bd4a9f56e86a42c9/users/0bb14422779947f6a6efb0cd19bd1d3f/roles/e151ac51ab124ad69365c08a0316c4ea` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:13.269336 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1baca4-c6fc-41ff-865b-5f9e1b77c6d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66d57cf1e2814541bd4a9f56e86a42c9, user_id=0bb14422779947f6a6efb0cd19bd1d3f, role_id=e151ac51ab124ad69365c08a0316c4ea)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:13.273803 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1baca4-c6fc-41ff-865b-5f9e1b77c6d5 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:13.291298 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1460/2913] 10.222.0.62 () {60 vars in 1483 bytes} [Wed Sep 7 13:44:13 2022] PUT /identity/v3/projects/66d57cf1e2814541bd4a9f56e86a42c9/users/0bb14422779947f6a6efb0cd19bd1d3f/roles/e151ac51ab124ad69365c08a0316c4ea => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:44:21.578504 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa26977-8f9a-4c6e-8f69-172dc64ea8cd None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:21.579820 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa26977-8f9a-4c6e-8f69-172dc64ea8cd None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:21.580265 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa26977-8f9a-4c6e-8f69-172dc64ea8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:21.621072 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-5aa26977-8f9a-4c6e-8f69-172dc64ea8cd None None] MFA Rules not processed for user `0bb14422779947f6a6efb0cd19bd1d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:44:21.622345 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-5aa26977-8f9a-4c6e-8f69-172dc64ea8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:44:21.761905 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1454/2914] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:44:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:44:21.777394 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1baca4-c6fc-41ff-865b-5f9e1b77c6d5 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:21.864948 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1baca4-c6fc-41ff-865b-5f9e1b77c6d5 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:21.875008 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f3bb0-1726-467d-93a6-427dc45e9c70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97c363046db444cbb3d4346066f9ad42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:21.876106 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f3bb0-1726-467d-93a6-427dc45e9c70 service nova] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:21.876602 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f3bb0-1726-467d-93a6-427dc45e9c70 service nova] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:21.877129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f3bb0-1726-467d-93a6-427dc45e9c70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:21.949404 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f3bb0-1726-467d-93a6-427dc45e9c70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:21.951551 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f3bb0-1726-467d-93a6-427dc45e9c70 service nova] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:21.965366 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1461/2915] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:44:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:44:26.759057 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f04fd-6373-4cf1-ab0a-2dea157fb35a None None] REQUEST_METHOD: `POST` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:26.759679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f04fd-6373-4cf1-ab0a-2dea157fb35a None None] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:26.759679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f04fd-6373-4cf1-ab0a-2dea157fb35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:26.783087 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.auth.core [None req-7f2f04fd-6373-4cf1-ab0a-2dea157fb35a None None] MFA Rules not processed for user `138facb7768545999dd43217600771a3`. Rule list: `[]` (Enabled: `True`). {{(pid=105315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:44:26.784348 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-7f2f04fd-6373-4cf1-ab0a-2dea157fb35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:44:26.793794 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1455/2916] 10.222.0.62 () {58 vars in 958 bytes} [Wed Sep 7 13:44:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:44:31.958727 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=73,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:44:36.775961 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:44:43.375996 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f3bb0-1726-467d-93a6-427dc45e9c70 service nova] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:44:43.440301 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f3bb0-1726-467d-93a6-427dc45e9c70 service nova] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:44:43.444272 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab1306-aa69-4e46-99d1-9e5d68683f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:44:43.445463 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab1306-aa69-4e46-99d1-9e5d68683f4e service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:44:43.445793 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab1306-aa69-4e46-99d1-9e5d68683f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:44:43.446035 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab1306-aa69-4e46-99d1-9e5d68683f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:44:43.447939 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-c5ab1306-aa69-4e46-99d1-9e5d68683f4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:44:43.458518 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab1306-aa69-4e46-99d1-9e5d68683f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:44:43.460004 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab1306-aa69-4e46-99d1-9e5d68683f4e service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:44:43.470360 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1462/2917] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:44:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:44:53.466403 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=26 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:45:03.509010 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f04fd-6373-4cf1-ab0a-2dea157fb35a None None] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:45:03.516776 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f04fd-6373-4cf1-ab0a-2dea157fb35a None None] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:45:03.520350 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de66c0e-4608-4832-9d46-80d6140f759e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:45:03.521017 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de66c0e-4608-4832-9d46-80d6140f759e service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:45:03.521298 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de66c0e-4608-4832-9d46-80d6140f759e service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:45:03.521568 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de66c0e-4608-4832-9d46-80d6140f759e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:45:03.532865 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de66c0e-4608-4832-9d46-80d6140f759e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:45:03.534398 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de66c0e-4608-4832-9d46-80d6140f759e service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:45:03.545593 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1456/2918] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:45:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:45:13.540927 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=6 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:45:26.748596 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ed2c6-4ea9-48f3-ba6d-a3ca63077797 None None] REQUEST_METHOD: `POST` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:45:26.749179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ed2c6-4ea9-48f3-ba6d-a3ca63077797 None None] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:45:26.749179 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ed2c6-4ea9-48f3-ba6d-a3ca63077797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:45:26.769181 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.auth.core [None req-281ed2c6-4ea9-48f3-ba6d-a3ca63077797 None None] MFA Rules not processed for user `8121ba4cf2cf44d9a5a55975ea85928c`. Rule list: `[]` (Enabled: `True`). {{(pid=105316) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 13:45:26.770284 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-281ed2c6-4ea9-48f3-ba6d-a3ca63077797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:45:26.835059 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1463/2919] 10.222.0.62 () {58 vars in 967 bytes} [Wed Sep 7 13:45:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 13:45:26.995867 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de66c0e-4608-4832-9d46-80d6140f759e service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:45:26.997465 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-3de66c0e-4608-4832-9d46-80d6140f759e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:45:27.094779 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de66c0e-4608-4832-9d46-80d6140f759e service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:45:27.100079 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101eff5-fe6c-48e6-a220-092b4fa7c0c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cedc7091d2f46b5be68818d26399e9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:45:27.100710 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101eff5-fe6c-48e6-a220-092b4fa7c0c5 service swift] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:45:27.100976 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101eff5-fe6c-48e6-a220-092b4fa7c0c5 service swift] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:45:27.101251 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101eff5-fe6c-48e6-a220-092b4fa7c0c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:45:27.111721 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101eff5-fe6c-48e6-a220-092b4fa7c0c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:45:27.113314 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101eff5-fe6c-48e6-a220-092b4fa7c0c5 service swift] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:45:27.122532 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1457/2920] 10.222.0.62 () {60 vars in 1384 bytes} [Wed Sep 7 13:45:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 13:45:36.827807 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=25 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:45:36.903438 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ed2c6-4ea9-48f3-ba6d-a3ca63077797 None None] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:45:36.905641 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-281ed2c6-4ea9-48f3-ba6d-a3ca63077797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:45:36.998329 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ed2c6-4ea9-48f3-ba6d-a3ca63077797 None None] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:45:37.003434 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd8c65-9289-491c-b9ce-b0d10c100cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:45:37.005009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd8c65-9289-491c-b9ce-b0d10c100cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:45:37.005009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd8c65-9289-491c-b9ce-b0d10c100cef admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:45:37.005009 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd8c65-9289-491c-b9ce-b0d10c100cef admin admin] PATH_INFO: `/v3/users/8895cee0f4444866bd4c3908eb248543` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:45:37.006514 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd8c65-9289-491c-b9ce-b0d10c100cef admin admin] RBAC: Authorizing `identity:delete_user(user_id=8895cee0f4444866bd4c3908eb248543)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:45:37.011137 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd8c65-9289-491c-b9ce-b0d10c100cef admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:45:37.118997 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=32 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:45:37.410321 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f6cd8c65-9289-491c-b9ce-b0d10c100cef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8895cee0f4444866bd4c3908eb248543'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:37.423938 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f6cd8c65-9289-491c-b9ce-b0d10c100cef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8895cee0f4444866bd4c3908eb248543'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:37.424737 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f6cd8c65-9289-491c-b9ce-b0d10c100cef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8895cee0f4444866bd4c3908eb248543'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:37.435188 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f6cd8c65-9289-491c-b9ce-b0d10c100cef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8895cee0f4444866bd4c3908eb248543'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:37.436113 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f6cd8c65-9289-491c-b9ce-b0d10c100cef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8895cee0f4444866bd4c3908eb248543'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:37.592435 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1464/2921] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:45:36 2022] DELETE /identity/v3/users/8895cee0f4444866bd4c3908eb248543 => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:45:37.968627 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101eff5-fe6c-48e6-a220-092b4fa7c0c5 service swift] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:45:37.980734 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101eff5-fe6c-48e6-a220-092b4fa7c0c5 service swift] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:45:38.054484 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66901e-c0eb-4d22-a595-c375b49b2162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:45:38.055706 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66901e-c0eb-4d22-a595-c375b49b2162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:45:38.056019 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66901e-c0eb-4d22-a595-c375b49b2162 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:45:38.056360 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66901e-c0eb-4d22-a595-c375b49b2162 admin admin] PATH_INFO: `/v3/projects/b933078b01aa4e3b82e37fef9b6ff92c` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:45:38.057752 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66901e-c0eb-4d22-a595-c375b49b2162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b933078b01aa4e3b82e37fef9b6ff92c)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:45:38.059969 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66901e-c0eb-4d22-a595-c375b49b2162 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:45:38.208154 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0e66901e-c0eb-4d22-a595-c375b49b2162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b933078b01aa4e3b82e37fef9b6ff92c', 'request_id': 'req-0e66901e-c0eb-4d22-a595-c375b49b2162'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:38.216312 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0e66901e-c0eb-4d22-a595-c375b49b2162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b933078b01aa4e3b82e37fef9b6ff92c', 'request_id': 'req-0e66901e-c0eb-4d22-a595-c375b49b2162'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:38.313148 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0e66901e-c0eb-4d22-a595-c375b49b2162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b933078b01aa4e3b82e37fef9b6ff92c', 'request_id': 'req-0e66901e-c0eb-4d22-a595-c375b49b2162'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:38.327697 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0e66901e-c0eb-4d22-a595-c375b49b2162 admin admin] The token cache is being invalidate because project b933078b01aa4e3b82e37fef9b6ff92c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:45:38.328070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0e66901e-c0eb-4d22-a595-c375b49b2162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:38.329081 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-0e66901e-c0eb-4d22-a595-c375b49b2162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:38.331882 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1458/2922] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:45:37 2022] DELETE /identity/v3/projects/b933078b01aa4e3b82e37fef9b6ff92c => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:45:38.343941 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd8c65-9289-491c-b9ce-b0d10c100cef admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:45:38.345524 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f6cd8c65-9289-491c-b9ce-b0d10c100cef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:45:38.359372 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd8c65-9289-491c-b9ce-b0d10c100cef admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:45:38.427641 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd55bf-04ad-4c8c-b319-518065ed21e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:45:38.428296 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd55bf-04ad-4c8c-b319-518065ed21e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:45:38.428605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd55bf-04ad-4c8c-b319-518065ed21e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:45:38.428870 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd55bf-04ad-4c8c-b319-518065ed21e4 admin admin] PATH_INFO: `/v3/users/50c2f9a37b674adc85e943ee3bf60b50` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:45:38.430794 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd55bf-04ad-4c8c-b319-518065ed21e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50c2f9a37b674adc85e943ee3bf60b50)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:45:38.433004 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd55bf-04ad-4c8c-b319-518065ed21e4 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:45:38.458439 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:45:38.733122 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-49cd55bf-04ad-4c8c-b319-518065ed21e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50c2f9a37b674adc85e943ee3bf60b50'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:38.746808 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-49cd55bf-04ad-4c8c-b319-518065ed21e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50c2f9a37b674adc85e943ee3bf60b50'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:38.747617 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-49cd55bf-04ad-4c8c-b319-518065ed21e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50c2f9a37b674adc85e943ee3bf60b50'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:38.755622 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-49cd55bf-04ad-4c8c-b319-518065ed21e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50c2f9a37b674adc85e943ee3bf60b50'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:38.756645 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-49cd55bf-04ad-4c8c-b319-518065ed21e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50c2f9a37b674adc85e943ee3bf60b50'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:38.873210 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1465/2923] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:45:38 2022] DELETE /identity/v3/users/50c2f9a37b674adc85e943ee3bf60b50 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:45:39.181018 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66901e-c0eb-4d22-a595-c375b49b2162 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:45:39.182842 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-0e66901e-c0eb-4d22-a595-c375b49b2162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:45:39.203860 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd55bf-04ad-4c8c-b319-518065ed21e4 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:45:39.206202 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-49cd55bf-04ad-4c8c-b319-518065ed21e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:45:39.297842 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66901e-c0eb-4d22-a595-c375b49b2162 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:45:39.342891 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd55bf-04ad-4c8c-b319-518065ed21e4 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:45:39.347929 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:45:39.349784 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac20fa-222c-41af-967c-69e13ed690e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:45:39.350653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac20fa-222c-41af-967c-69e13ed690e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:45:39.351396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac20fa-222c-41af-967c-69e13ed690e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:45:39.351396 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac20fa-222c-41af-967c-69e13ed690e3 admin admin] PATH_INFO: `/v3/projects/e1c30057405b424b8ed90e2a9e518f12` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:45:39.365421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac20fa-222c-41af-967c-69e13ed690e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1c30057405b424b8ed90e2a9e518f12)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:45:39.365421 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac20fa-222c-41af-967c-69e13ed690e3 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:45:39.396285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0351747-9cd6-46d8-ae9b-8543f4f1f062 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bee90bf951f04c2e8e19ceeacb8cfcb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:45:39.397238 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0351747-9cd6-46d8-ae9b-8543f4f1f062 service placement] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:45:39.397718 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0351747-9cd6-46d8-ae9b-8543f4f1f062 service placement] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:45:39.398191 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0351747-9cd6-46d8-ae9b-8543f4f1f062 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:45:39.402494 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f0351747-9cd6-46d8-ae9b-8543f4f1f062 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:45:39.422227 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0351747-9cd6-46d8-ae9b-8543f4f1f062 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:45:39.424008 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0351747-9cd6-46d8-ae9b-8543f4f1f062 service placement] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:45:39.435783 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1459/2924] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:45:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:45:39.489463 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-21ac20fa-222c-41af-967c-69e13ed690e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1c30057405b424b8ed90e2a9e518f12', 'request_id': 'req-21ac20fa-222c-41af-967c-69e13ed690e3'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:39.496988 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-21ac20fa-222c-41af-967c-69e13ed690e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1c30057405b424b8ed90e2a9e518f12', 'request_id': 'req-21ac20fa-222c-41af-967c-69e13ed690e3'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:39.610011 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-21ac20fa-222c-41af-967c-69e13ed690e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1c30057405b424b8ed90e2a9e518f12', 'request_id': 'req-21ac20fa-222c-41af-967c-69e13ed690e3'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:39.623790 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-21ac20fa-222c-41af-967c-69e13ed690e3 admin admin] The token cache is being invalidate because project e1c30057405b424b8ed90e2a9e518f12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:45:39.624096 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-21ac20fa-222c-41af-967c-69e13ed690e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:39.624799 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-21ac20fa-222c-41af-967c-69e13ed690e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:45:39.626375 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1466/2925] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:45:39 2022] DELETE /identity/v3/projects/e1c30057405b424b8ed90e2a9e518f12 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:45:49.620283 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:DELETE=14,keystone:SELECT=71,keystone:INSERT=2 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:45:49.638356 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=17 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:46:11.582161 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0351747-9cd6-46d8-ae9b-8543f4f1f062 service placement] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:46:11.583696 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-f0351747-9cd6-46d8-ae9b-8543f4f1f062 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:46:11.652266 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0351747-9cd6-46d8-ae9b-8543f4f1f062 service placement] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:46:11.707323 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9522742-2390-4932-9dec-05e786ae7115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:46:11.707866 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9522742-2390-4932-9dec-05e786ae7115 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:46:11.708127 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9522742-2390-4932-9dec-05e786ae7115 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:46:11.708402 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9522742-2390-4932-9dec-05e786ae7115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:46:11.710231 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c9522742-2390-4932-9dec-05e786ae7115 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:46:11.781013 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9522742-2390-4932-9dec-05e786ae7115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:46:11.782481 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9522742-2390-4932-9dec-05e786ae7115 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:46:11.794299 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1460/2926] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:46:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:46:15.509810 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac20fa-222c-41af-967c-69e13ed690e3 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:46:15.511826 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-21ac20fa-222c-41af-967c-69e13ed690e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:46:15.618858 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac20fa-222c-41af-967c-69e13ed690e3 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:46:15.623890 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61b015-dcd8-4a37-b0d8-4a228359e376 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bee90bf951f04c2e8e19ceeacb8cfcb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:46:15.626040 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61b015-dcd8-4a37-b0d8-4a228359e376 service placement] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:46:15.626040 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61b015-dcd8-4a37-b0d8-4a228359e376 service placement] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:46:15.626040 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61b015-dcd8-4a37-b0d8-4a228359e376 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:46:15.627464 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-fc61b015-dcd8-4a37-b0d8-4a228359e376 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:46:15.641139 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61b015-dcd8-4a37-b0d8-4a228359e376 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:46:15.642894 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61b015-dcd8-4a37-b0d8-4a228359e376 service placement] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:46:15.653974 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1467/2927] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:46:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:46:20.420224 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9522742-2390-4932-9dec-05e786ae7115 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:46:20.422494 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-c9522742-2390-4932-9dec-05e786ae7115 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:46:20.435055 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9522742-2390-4932-9dec-05e786ae7115 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:46:20.441185 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f044a5-35c3-4181-a835-5d97fbf3ebe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:46:20.441957 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f044a5-35c3-4181-a835-5d97fbf3ebe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:46:20.442133 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f044a5-35c3-4181-a835-5d97fbf3ebe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:46:20.442416 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f044a5-35c3-4181-a835-5d97fbf3ebe5 admin admin] PATH_INFO: `/v3/users/9dae215cd92f46d18a3c1b7389c3af4b` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:46:20.443635 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f044a5-35c3-4181-a835-5d97fbf3ebe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dae215cd92f46d18a3c1b7389c3af4b)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:46:20.445557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f044a5-35c3-4181-a835-5d97fbf3ebe5 admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:46:20.674595 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-09f044a5-35c3-4181-a835-5d97fbf3ebe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dae215cd92f46d18a3c1b7389c3af4b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:20.798011 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-09f044a5-35c3-4181-a835-5d97fbf3ebe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dae215cd92f46d18a3c1b7389c3af4b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:20.798967 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-09f044a5-35c3-4181-a835-5d97fbf3ebe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dae215cd92f46d18a3c1b7389c3af4b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:20.799934 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-09f044a5-35c3-4181-a835-5d97fbf3ebe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dae215cd92f46d18a3c1b7389c3af4b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:20.809095 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:46:20.811428 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-09f044a5-35c3-4181-a835-5d97fbf3ebe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dae215cd92f46d18a3c1b7389c3af4b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:20.830897 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1461/2928] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:46:20 2022] DELETE /identity/v3/users/9dae215cd92f46d18a3c1b7389c3af4b => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:46:21.131965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61b015-dcd8-4a37-b0d8-4a228359e376 service placement] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:46:21.146006 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61b015-dcd8-4a37-b0d8-4a228359e376 service placement] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:46:21.230658 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40311264-c441-462c-ba08-b6d7b27b3562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:46:21.230929 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40311264-c441-462c-ba08-b6d7b27b3562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:46:21.231203 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40311264-c441-462c-ba08-b6d7b27b3562 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:46:21.231479 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40311264-c441-462c-ba08-b6d7b27b3562 admin admin] PATH_INFO: `/v3/projects/b4d5ce9b4fe34fc094bdacc3ca994537` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:46:21.232674 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40311264-c441-462c-ba08-b6d7b27b3562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4d5ce9b4fe34fc094bdacc3ca994537)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:46:21.235223 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40311264-c441-462c-ba08-b6d7b27b3562 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:46:21.357852 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-40311264-c441-462c-ba08-b6d7b27b3562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4d5ce9b4fe34fc094bdacc3ca994537', 'request_id': 'req-40311264-c441-462c-ba08-b6d7b27b3562'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:21.365426 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-40311264-c441-462c-ba08-b6d7b27b3562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4d5ce9b4fe34fc094bdacc3ca994537', 'request_id': 'req-40311264-c441-462c-ba08-b6d7b27b3562'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:21.437373 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-40311264-c441-462c-ba08-b6d7b27b3562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4d5ce9b4fe34fc094bdacc3ca994537', 'request_id': 'req-40311264-c441-462c-ba08-b6d7b27b3562'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:21.449464 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-40311264-c441-462c-ba08-b6d7b27b3562 admin admin] The token cache is being invalidate because project b4d5ce9b4fe34fc094bdacc3ca994537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:46:21.449754 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-40311264-c441-462c-ba08-b6d7b27b3562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:21.450466 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-40311264-c441-462c-ba08-b6d7b27b3562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:21.452034 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1468/2929] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:46:21 2022] DELETE /identity/v3/projects/b4d5ce9b4fe34fc094bdacc3ca994537 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 13:46:21.457353 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f044a5-35c3-4181-a835-5d97fbf3ebe5 admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:46:21.458996 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-09f044a5-35c3-4181-a835-5d97fbf3ebe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:46:21.473311 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f044a5-35c3-4181-a835-5d97fbf3ebe5 admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:46:21.545203 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd580e-ca6f-4076-a8d5-cdaa82a1777b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:46:21.546391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd580e-ca6f-4076-a8d5-cdaa82a1777b admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:46:21.546391 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd580e-ca6f-4076-a8d5-cdaa82a1777b admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:46:21.547208 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd580e-ca6f-4076-a8d5-cdaa82a1777b admin admin] PATH_INFO: `/v3/users/1b7d4a431f1c4feca7d33fcdb8ec1b7b` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:46:21.548571 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd580e-ca6f-4076-a8d5-cdaa82a1777b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b7d4a431f1c4feca7d33fcdb8ec1b7b)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:46:21.550555 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd580e-ca6f-4076-a8d5-cdaa82a1777b admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:46:21.749793 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-dbbd580e-ca6f-4076-a8d5-cdaa82a1777b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b7d4a431f1c4feca7d33fcdb8ec1b7b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:21.825900 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-dbbd580e-ca6f-4076-a8d5-cdaa82a1777b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b7d4a431f1c4feca7d33fcdb8ec1b7b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:21.826630 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-dbbd580e-ca6f-4076-a8d5-cdaa82a1777b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b7d4a431f1c4feca7d33fcdb8ec1b7b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:21.827479 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-dbbd580e-ca6f-4076-a8d5-cdaa82a1777b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b7d4a431f1c4feca7d33fcdb8ec1b7b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:21.837375 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-dbbd580e-ca6f-4076-a8d5-cdaa82a1777b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b7d4a431f1c4feca7d33fcdb8ec1b7b'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:21.870124 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1462/2930] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:46:21 2022] DELETE /identity/v3/users/1b7d4a431f1c4feca7d33fcdb8ec1b7b => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:46:22.174629 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40311264-c441-462c-ba08-b6d7b27b3562 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:46:22.177241 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-40311264-c441-462c-ba08-b6d7b27b3562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:46:22.194197 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40311264-c441-462c-ba08-b6d7b27b3562 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:46:22.213823 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:46:22.255070 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768dfed-7cb2-4556-a230-04d7cc9cbc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:46:22.255657 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768dfed-7cb2-4556-a230-04d7cc9cbc88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:46:22.255936 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768dfed-7cb2-4556-a230-04d7cc9cbc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:46:22.256212 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768dfed-7cb2-4556-a230-04d7cc9cbc88 admin admin] PATH_INFO: `/v3/projects/605c6eb7c686455998808c1c39edb8cb` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:46:22.257454 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768dfed-7cb2-4556-a230-04d7cc9cbc88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=605c6eb7c686455998808c1c39edb8cb)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:46:22.259414 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768dfed-7cb2-4556-a230-04d7cc9cbc88 admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:46:22.379981 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f768dfed-7cb2-4556-a230-04d7cc9cbc88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '605c6eb7c686455998808c1c39edb8cb', 'request_id': 'req-f768dfed-7cb2-4556-a230-04d7cc9cbc88'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:22.387813 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f768dfed-7cb2-4556-a230-04d7cc9cbc88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '605c6eb7c686455998808c1c39edb8cb', 'request_id': 'req-f768dfed-7cb2-4556-a230-04d7cc9cbc88'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:22.471220 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f768dfed-7cb2-4556-a230-04d7cc9cbc88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '605c6eb7c686455998808c1c39edb8cb', 'request_id': 'req-f768dfed-7cb2-4556-a230-04d7cc9cbc88'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:22.485246 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f768dfed-7cb2-4556-a230-04d7cc9cbc88 admin admin] The token cache is being invalidate because project 605c6eb7c686455998808c1c39edb8cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105316) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:46:22.485644 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f768dfed-7cb2-4556-a230-04d7cc9cbc88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:22.486653 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-f768dfed-7cb2-4556-a230-04d7cc9cbc88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:46:22.488684 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1469/2931] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:46:22 2022] DELETE /identity/v3/projects/605c6eb7c686455998808c1c39edb8cb => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:46:31.865240 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=61,keystone:DELETE=12,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:46:32.481142 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:47:18.697932 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd580e-ca6f-4076-a8d5-cdaa82a1777b admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:47:18.708975 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-dbbd580e-ca6f-4076-a8d5-cdaa82a1777b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:47:18.768570 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd580e-ca6f-4076-a8d5-cdaa82a1777b admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:47:18.822175 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d3efd-19ea-4181-bfba-eb3a0ffa549d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:47:18.822807 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d3efd-19ea-4181-bfba-eb3a0ffa549d service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:47:18.823125 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d3efd-19ea-4181-bfba-eb3a0ffa549d service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:47:18.823397 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d3efd-19ea-4181-bfba-eb3a0ffa549d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:47:18.825165 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-482d3efd-19ea-4181-bfba-eb3a0ffa549d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:47:18.836420 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d3efd-19ea-4181-bfba-eb3a0ffa549d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:47:18.837854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d3efd-19ea-4181-bfba-eb3a0ffa549d service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:47:18.847216 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1463/2932] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:47:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:47:18.926605 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768dfed-7cb2-4556-a230-04d7cc9cbc88 admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:47:18.928227 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-f768dfed-7cb2-4556-a230-04d7cc9cbc88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:47:18.940794 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768dfed-7cb2-4556-a230-04d7cc9cbc88 admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:47:18.944900 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f73f54-5992-49fe-bf60-2163d8dc5bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:47:18.945505 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f73f54-5992-49fe-bf60-2163d8dc5bdf service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:47:18.945780 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f73f54-5992-49fe-bf60-2163d8dc5bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:47:18.946052 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f73f54-5992-49fe-bf60-2163d8dc5bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:47:18.947763 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-12f73f54-5992-49fe-bf60-2163d8dc5bdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:47:19.026494 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f73f54-5992-49fe-bf60-2163d8dc5bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:47:19.028129 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f73f54-5992-49fe-bf60-2163d8dc5bdf service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:47:19.039918 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1470/2933] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:47:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 13:47:28.842461 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=46 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:47:29.034308 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=34 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:48:34.849949 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d3efd-19ea-4181-bfba-eb3a0ffa549d service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:48:34.851443 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-482d3efd-19ea-4181-bfba-eb3a0ffa549d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:48:34.864048 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d3efd-19ea-4181-bfba-eb3a0ffa549d service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:48:34.868233 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84832048-a132-45b6-a69f-5296bbef5c89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:48:34.868854 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84832048-a132-45b6-a69f-5296bbef5c89 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:48:34.869566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84832048-a132-45b6-a69f-5296bbef5c89 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:48:34.869566 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84832048-a132-45b6-a69f-5296bbef5c89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:48:34.871487 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-84832048-a132-45b6-a69f-5296bbef5c89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:48:34.882181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84832048-a132-45b6-a69f-5296bbef5c89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:48:34.883651 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84832048-a132-45b6-a69f-5296bbef5c89 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:48:34.892176 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1464/2934] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:48:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:48:44.888461 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=6 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:48:48.029154 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f73f54-5992-49fe-bf60-2163d8dc5bdf service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:48:48.038738 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f73f54-5992-49fe-bf60-2163d8dc5bdf service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:48:48.042545 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff0317-a30f-479d-957e-652357cec294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:48:48.043186 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff0317-a30f-479d-957e-652357cec294 service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:48:48.043470 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff0317-a30f-479d-957e-652357cec294 service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:48:48.043742 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff0317-a30f-479d-957e-652357cec294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:48:48.045758 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-fdff0317-a30f-479d-957e-652357cec294 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:48:48.084575 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff0317-a30f-479d-957e-652357cec294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:48:48.085933 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff0317-a30f-479d-957e-652357cec294 service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:48:48.100449 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1471/2935] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:48:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:48:49.311162 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84832048-a132-45b6-a69f-5296bbef5c89 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:48:49.320382 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84832048-a132-45b6-a69f-5296bbef5c89 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:48:49.324354 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d84d89-f703-4404-936a-769fb7db1f35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:48:49.324956 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d84d89-f703-4404-936a-769fb7db1f35 service neutron] REQUEST_METHOD: `GET` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:48:49.325234 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d84d89-f703-4404-936a-769fb7db1f35 service neutron] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:48:49.325501 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d84d89-f703-4404-936a-769fb7db1f35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:48:49.327285 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-44d84d89-f703-4404-936a-769fb7db1f35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:48:49.342860 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d84d89-f703-4404-936a-769fb7db1f35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:48:49.344359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d84d89-f703-4404-936a-769fb7db1f35 service neutron] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:48:49.357529 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1465/2936] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:48:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:48:57.028446 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff0317-a30f-479d-957e-652357cec294 service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:48:57.031033 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-fdff0317-a30f-479d-957e-652357cec294 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:48:57.143916 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff0317-a30f-479d-957e-652357cec294 service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:48:57.149989 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaffe39-46e0-490e-b03c-50f1f21567df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bee90bf951f04c2e8e19ceeacb8cfcb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:48:57.150656 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaffe39-46e0-490e-b03c-50f1f21567df service placement] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:48:57.151003 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaffe39-46e0-490e-b03c-50f1f21567df service placement] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:48:57.151293 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaffe39-46e0-490e-b03c-50f1f21567df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:48:57.153103 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-afaffe39-46e0-490e-b03c-50f1f21567df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:48:57.165791 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaffe39-46e0-490e-b03c-50f1f21567df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:48:57.167416 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaffe39-46e0-490e-b03c-50f1f21567df service placement] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:48:57.182889 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1472/2937] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:48:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:48:59.351886 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=6 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:49:01.565581 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d84d89-f703-4404-936a-769fb7db1f35 service neutron] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:49:01.567809 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-44d84d89-f703-4404-936a-769fb7db1f35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:49:01.694385 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d84d89-f703-4404-936a-769fb7db1f35 service neutron] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:49:01.699164 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a6845-150f-4c0b-ae40-358b0cf6af0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:49:01.699784 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a6845-150f-4c0b-ae40-358b0cf6af0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:49:01.700062 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a6845-150f-4c0b-ae40-358b0cf6af0a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:49:01.700359 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a6845-150f-4c0b-ae40-358b0cf6af0a admin admin] PATH_INFO: `/v3/users/138facb7768545999dd43217600771a3` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:49:01.701510 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a6845-150f-4c0b-ae40-358b0cf6af0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=138facb7768545999dd43217600771a3)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:49:01.703463 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a6845-150f-4c0b-ae40-358b0cf6af0a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:49:01.990736 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e32a6845-150f-4c0b-ae40-358b0cf6af0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '138facb7768545999dd43217600771a3'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:02.110947 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e32a6845-150f-4c0b-ae40-358b0cf6af0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '138facb7768545999dd43217600771a3'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:02.111695 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e32a6845-150f-4c0b-ae40-358b0cf6af0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '138facb7768545999dd43217600771a3'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:02.112337 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e32a6845-150f-4c0b-ae40-358b0cf6af0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '138facb7768545999dd43217600771a3'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:02.120385 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-e32a6845-150f-4c0b-ae40-358b0cf6af0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '138facb7768545999dd43217600771a3'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:02.138365 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1466/2938] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:49:01 2022] DELETE /identity/v3/users/138facb7768545999dd43217600771a3 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:49:02.149208 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaffe39-46e0-490e-b03c-50f1f21567df service placement] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:49:02.160443 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaffe39-46e0-490e-b03c-50f1f21567df service placement] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:49:02.213841 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab6aff-7843-4e42-98ac-4a948020e80b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acdbdb6f23b4f10bcbab0387ea67ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:49:02.214509 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab6aff-7843-4e42-98ac-4a948020e80b service neutron] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:49:02.214787 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab6aff-7843-4e42-98ac-4a948020e80b service neutron] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:49:02.215092 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab6aff-7843-4e42-98ac-4a948020e80b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:49:02.216878 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-19ab6aff-7843-4e42-98ac-4a948020e80b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:49:02.259305 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=100 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:49:02.287927 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab6aff-7843-4e42-98ac-4a948020e80b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:49:02.289549 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab6aff-7843-4e42-98ac-4a948020e80b service neutron] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:49:02.299965 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1473/2939] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:49:02.592667 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a6845-150f-4c0b-ae40-358b0cf6af0a admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:49:02.597357 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-e32a6845-150f-4c0b-ae40-358b0cf6af0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:49:02.611172 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a6845-150f-4c0b-ae40-358b0cf6af0a admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:49:02.618831 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b03017-11f1-4cd1-a99b-c4004425962a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:49:02.620416 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b03017-11f1-4cd1-a99b-c4004425962a admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:49:02.621070 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b03017-11f1-4cd1-a99b-c4004425962a admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:49:02.621384 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b03017-11f1-4cd1-a99b-c4004425962a admin admin] PATH_INFO: `/v3/projects/bb7b613bf3c94b4aa49f450be9971774` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:49:02.622644 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b03017-11f1-4cd1-a99b-c4004425962a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb7b613bf3c94b4aa49f450be9971774)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:49:02.624643 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b03017-11f1-4cd1-a99b-c4004425962a admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:49:02.652531 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:49:02.791000 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b8b03017-11f1-4cd1-a99b-c4004425962a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb7b613bf3c94b4aa49f450be9971774', 'request_id': 'req-b8b03017-11f1-4cd1-a99b-c4004425962a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:02.797638 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b8b03017-11f1-4cd1-a99b-c4004425962a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb7b613bf3c94b4aa49f450be9971774', 'request_id': 'req-b8b03017-11f1-4cd1-a99b-c4004425962a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:02.909031 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b8b03017-11f1-4cd1-a99b-c4004425962a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb7b613bf3c94b4aa49f450be9971774', 'request_id': 'req-b8b03017-11f1-4cd1-a99b-c4004425962a'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:02.921811 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b8b03017-11f1-4cd1-a99b-c4004425962a admin admin] The token cache is being invalidate because project bb7b613bf3c94b4aa49f450be9971774 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:49:02.922181 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b8b03017-11f1-4cd1-a99b-c4004425962a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:02.923049 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b8b03017-11f1-4cd1-a99b-c4004425962a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:02.924557 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1467/2940] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:49:02 2022] DELETE /identity/v3/projects/bb7b613bf3c94b4aa49f450be9971774 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:49:02.928769 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab6aff-7843-4e42-98ac-4a948020e80b service neutron] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:49:02.942553 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab6aff-7843-4e42-98ac-4a948020e80b service neutron] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:49:03.001521 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75830a9c-5af3-40d5-a5a6-480f5ebb31ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:49:03.002152 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75830a9c-5af3-40d5-a5a6-480f5ebb31ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:49:03.002461 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75830a9c-5af3-40d5-a5a6-480f5ebb31ac admin admin] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:49:03.002741 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75830a9c-5af3-40d5-a5a6-480f5ebb31ac admin admin] PATH_INFO: `/v3/users/0bb14422779947f6a6efb0cd19bd1d3f` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:49:03.003901 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75830a9c-5af3-40d5-a5a6-480f5ebb31ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bb14422779947f6a6efb0cd19bd1d3f)` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:49:03.005796 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75830a9c-5af3-40d5-a5a6-480f5ebb31ac admin admin] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:49:03.382626 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-75830a9c-5af3-40d5-a5a6-480f5ebb31ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bb14422779947f6a6efb0cd19bd1d3f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:03.396911 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-75830a9c-5af3-40d5-a5a6-480f5ebb31ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bb14422779947f6a6efb0cd19bd1d3f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:03.397682 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-75830a9c-5af3-40d5-a5a6-480f5ebb31ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bb14422779947f6a6efb0cd19bd1d3f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:03.406115 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-75830a9c-5af3-40d5-a5a6-480f5ebb31ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bb14422779947f6a6efb0cd19bd1d3f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:03.406797 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.notifications [None req-75830a9c-5af3-40d5-a5a6-480f5ebb31ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bb14422779947f6a6efb0cd19bd1d3f'} {{(pid=105316) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:03.536482 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1474/2941] 10.222.0.62 () {58 vars in 1224 bytes} [Wed Sep 7 13:49:02 2022] DELETE /identity/v3/users/0bb14422779947f6a6efb0cd19bd1d3f => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:49:03.846286 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b03017-11f1-4cd1-a99b-c4004425962a admin admin] Authenticating user token {{(pid=105315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:49:03.847773 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.fernet_utils [None req-b8b03017-11f1-4cd1-a99b-c4004425962a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:49:03.861903 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b03017-11f1-4cd1-a99b-c4004425962a admin admin] Validating token access rules against request {{(pid=105315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:49:03.924437 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6116601-07b2-4f18-beb5-1ff8d85066ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a8e484a77ce40f880d21a33a930f0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ee134892b8643809999afc672e86085', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:49:03.924978 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6116601-07b2-4f18-beb5-1ff8d85066ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:49:03.925256 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6116601-07b2-4f18-beb5-1ff8d85066ed admin admin] SCRIPT_NAME: `/identity` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:49:03.925524 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6116601-07b2-4f18-beb5-1ff8d85066ed admin admin] PATH_INFO: `/v3/projects/66d57cf1e2814541bd4a9f56e86a42c9` {{(pid=105315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:49:03.926679 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6116601-07b2-4f18-beb5-1ff8d85066ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=66d57cf1e2814541bd4a9f56e86a42c9)` {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:49:03.928603 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6116601-07b2-4f18-beb5-1ff8d85066ed admin admin] RBAC: Authorization granted {{(pid=105315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:49:04.104104 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b6116601-07b2-4f18-beb5-1ff8d85066ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66d57cf1e2814541bd4a9f56e86a42c9', 'request_id': 'req-b6116601-07b2-4f18-beb5-1ff8d85066ed'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:04.112400 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b6116601-07b2-4f18-beb5-1ff8d85066ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66d57cf1e2814541bd4a9f56e86a42c9', 'request_id': 'req-b6116601-07b2-4f18-beb5-1ff8d85066ed'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:04.219307 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b6116601-07b2-4f18-beb5-1ff8d85066ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66d57cf1e2814541bd4a9f56e86a42c9', 'request_id': 'req-b6116601-07b2-4f18-beb5-1ff8d85066ed'} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:04.232457 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b6116601-07b2-4f18-beb5-1ff8d85066ed admin admin] The token cache is being invalidate because project 66d57cf1e2814541bd4a9f56e86a42c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=105315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 13:49:04.234096 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b6116601-07b2-4f18-beb5-1ff8d85066ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:04.234096 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG keystone.notifications [None req-b6116601-07b2-4f18-beb5-1ff8d85066ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=105315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 13:49:04.235064 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: [pid: 105315|app: 0|req: 1468/2942] 10.222.0.62 () {58 vars in 1233 bytes} [Wed Sep 7 13:49:03 2022] DELETE /identity/v3/projects/66d57cf1e2814541bd4a9f56e86a42c9 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 13:49:13.532114 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=63,keystone:DELETE=10,keystone:INSERT=1 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:49:14.228609 dv-cinder-856259-1-iscsi devstack@keystone.service[105315]: DEBUG dbcounter [-] [105315] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=105315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 13:50:43.157391 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75830a9c-5af3-40d5-a5a6-480f5ebb31ac admin admin] Authenticating user token {{(pid=105316) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 13:50:43.159218 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-75830a9c-5af3-40d5-a5a6-480f5ebb31ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:50:43.259133 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75830a9c-5af3-40d5-a5a6-480f5ebb31ac admin admin] Validating token access rules against request {{(pid=105316) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 13:50:43.323228 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e75c7-e236-4495-92b1-760745014f0e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bee90bf951f04c2e8e19ceeacb8cfcb2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4574cf3ff270485faf60594019f7e05c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=105316) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 13:50:43.325107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e75c7-e236-4495-92b1-760745014f0e service placement] REQUEST_METHOD: `GET` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 13:50:43.325107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e75c7-e236-4495-92b1-760745014f0e service placement] SCRIPT_NAME: `/identity` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 13:50:43.325107 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e75c7-e236-4495-92b1-760745014f0e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=105316) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 13:50:43.326536 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.fernet_utils [None req-be4e75c7-e236-4495-92b1-760745014f0e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=105316) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 13:50:43.338311 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e75c7-e236-4495-92b1-760745014f0e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 13:50:43.339928 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e75c7-e236-4495-92b1-760745014f0e service placement] RBAC: Authorization granted {{(pid=105316) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 13:50:43.351331 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: [pid: 105316|app: 0|req: 1475/2943] 10.222.0.62 () {60 vars in 1365 bytes} [Wed Sep 7 13:50:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 13:50:53.347195 dv-cinder-856259-1-iscsi devstack@keystone.service[105316]: DEBUG dbcounter [-] [105316] Writing DB stats keystone:SELECT=67 {{(pid=105316) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}